Poster

SoftSort: A Continuous Relaxation for the argsort Operator

Sebastian Prillo · Julian M Eisenschlos

Keywords: [ Algorithms ] [ Ranking and Preference Learning ] [ Deep Learning - Algorithms ]

[ Abstract ]
Tue 14 Jul 10 a.m. PDT — 10:45 a.m. PDT
Tue 14 Jul 10 p.m. PDT — 10:45 p.m. PDT

Abstract:

While sorting is an important procedure in computer science, the argsort operator - which takes as input a vector and returns its sorting permutation - has a discrete image and thus zero gradients almost everywhere. This prohibits end-to-end, gradient-based learning of models that rely on the argsort operator. A natural way to overcome this problem is to replace the argsort operator with a continuous relaxation. Recent work has shown a number of ways to do this, but the relaxations proposed so far are computationally complex. In this work we propose a simple continuous relaxation for the argsort operator which has the following qualities: it can be implemented in three lines of code, achieves state-of-the-art performance, is easy to reason about mathematically - substantially simplifying proofs - and is faster than competing approaches. We open source the code to reproduce all of the experiments and results.

Chat is not available.