Timezone: »

 
Spotlight
Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision
Felix Petersen · Christian Borgelt · Hilde Kuehne · Oliver Deussen

Tue Jul 20 05:35 PM -- 05:40 PM (PDT) @
in Deep Learning Algorithms 6 »

Sorting and ranking supervision is a method for training neural networks end-to-end based on ordering constraints. That is, the ground truth order of sets of samples is known, while their absolute values remain unsupervised. For that, we propose differentiable sorting networks by relaxing their pairwise conditional swap operations. To address the problems of vanishing gradients and extensive blurring that arise with larger numbers of layers, we propose mapping activations to regions with moderate gradients. We consider odd-even as well as bitonic sorting networks, which outperform existing relaxations of the sorting operation. We show that bitonic sorting networks can achieve stable training on large input sets of up to 1024 elements.

Keywords: Algorithms

Author Information

Felix Petersen (University of Konstanz)

Felix Petersen is a researcher and Ph.D. student in the Department of Computer Science at the University of Konstanz. His main research interests are investigating neural networks and combining them with differentiable algorithms, e.g., for solving unsupervised inverse problems. By the age of 19, Felix was the youngest student who obtained the degree of Bachelor of Computer Science one year after starting his Ph.D. research. In 2019, he was awarded the Konrad-Zuse-youth-price for extraordinary work in the domain of artificial intelligence. In the past, Felix has worked, i.a., at TAU, DESY, PSI, and CERN.

Christian Borgelt (University of Salzburg)
Hilde Kuehne (University of Frankfurt)
Oliver Deussen (University of Konstanz)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors