Gradients and derivatives are integral to machine learning, as they enable gradient-based optimization. In many real applications, however, models rest on algorithmic components that implement discrete decisions, or rely on discrete intermediate representations and structures. These discrete steps are intrinsically non-differentiable and accordingly break the flow of gradients. To use gradient-based approaches to learn the parameters of such models requires turning these non-differentiable components differentiable. This can be done with careful considerations, notably, using smoothing or relaxations to propose differentiable proxies for these components. With the advent of modular deep learning frameworks, these ideas have become more popular than ever in many fields of machine learning, generating in a short time-span a multitude of "differentiable everything", impacting topics as varied as rendering, sorting and ranking, convex optimizers, shortest-paths, dynamic programming, physics simulations, NN architecture search, top-k, graph algorithms, weakly- and self-supervised learning, and many more.
Opening Remarks (Remarks) | |
Invited Talk 1: Perturbed Optimizers for Learning (Invited Talk) | |
Invited Talk 2: Generalizing the Gumbel-Softmax with Stochastic Softmax Tricks (Invited Talk) | |
Coffee Break (Break) | |
Invited Talk 3: Differentiable Learning modulo Formal Verification (Invited Talk) | |
Short Poster Talks 1 (Short Poster Talks) | |
Poster Session 1 (Poster Session) | |
Lunch Break (Break) | |
Invited Talk 4: Blackbox Differentiation: the story so far (Invited Talk) | |
Invited Talk 5: On Differentiable Top-k Operators (Invited Talk) | |
Coffee Break (Break) | |
Short Poster Talks 2 (Short Poster Talks) | |
Invited Talk 6: Differentiable Rendering and Beyond (Invited Talk) | |
Closing Remarks (Remarks) | |
Poster Session 2 (Poster Session) | |
Differentiable MaxSAT Message Passing (Poster) | |
Dilated Convolution with Learnable Spacings: beyond bilinear interpolation (Poster) | |
Optimizing probability of barrier crossing with differentiable simulators (Poster) | |
EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search (Poster) | |
Towards Understanding Gradient Approximation in Equality Constrained Deep Declarative Networks (Poster) | |
PMaF: Deep Declarative Layers for Principal Matrix Features (Poster) | |
End-to-end Differentiable Clustering with Associative Memories (Poster) | |
Probabilistic Task-Adaptive Graph Rewiring (Poster) | |
Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information (Poster) | |
Differentiable Forward Projector for X-ray Computed Tomography (Poster) | |
Distributions for Compositionally Differentiating Parametric Discontinuities (Poster) | |
TaskMet: Task-Driven Metric Learning for Model Learning (Poster) | |
Dynamic Control of Queuing Networks via Differentiable Discrete-Event Simulation (Poster) | |
Stochastic Gradient Bayesian Optimal Experimental Designs for Simulation Based Inference (Poster) | |
Efficient Surrogate Gradients for Training Spiking Neural Networks (Poster) | |
Plateau-Reduced Differentiable Path Tracing (Poster) | |
Differentiable Tree Operations Promote Compositional Generalization (Poster) | |
GeoPhy: Differentiable Phylogenetic Inference via Geometric Gradients of Tree Topologies (Poster) | |
Differentiable Clustering and Partial Fenchel-Young Losses (Poster) | |
Latent Random Steps as Relaxations of Max-Cut, Min-Cut, and More (Poster) | |
PDP: Parameter-free Differentiable Pruning is All You Need (Poster) | |
Sample-efficient learning of auditory object representations using differentiable impulse response synthesis (Poster) | |
Some challenges of calibrating differentiable agent-based models (Poster) | |
Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning (Poster) | |
SelMix: Selective Mixup Fine Tuning for Optimizing Non-Decomposable Metrics (Poster) | |
Differentiating Metropolis-Hastings to Optimize Intractable Densities (Poster) | |
Lossless hardening with $\partial\mathbb{B}$ nets (Poster) | |
Learning Observation Models with Incremental Non-Differentiable Graph Optimizers in the Loop for Robotics State Estimation (Poster) | |
JAX FDM: A differentiable solver for inverse form-finding (Poster) | |
Fine-Tuning Language Models with Just Forward Passes (Poster) | |
SIMPLE: A Gradient Estimator for $k$-subset Sampling (Poster) | |
DNArch: Learning Convolutional Neural Architectures by Backpropagation (Poster) | |
A Unified Approach to Count-Based Weakly-Supervised Learning (Poster) | |
Interpretable Neural-Symbolic Concept Reasoning (Poster) | |
A Gradient Flow Modification to Improve Learning from Differentiable Quantum Simulators (Poster) | |
Investigating Axis-Aligned Differentiable Trees through Neural Tangent Kernels (Poster) | |
A Short Review of Automatic Differentiation Pitfalls in Scientific Computing (Poster) | |
Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models (Poster) | |
Koopman Constrained Policy Optimization: A Koopman operator theoretic method for differentiable optimal control in robotics (Poster) | |
Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trick (Poster) | |
Differentiable Set Partitioning (Poster) | |
From Perception to Programs: Regularize, Overparameterize, and Amortize (Poster) | |
Differentiable Search of Evolutionary Trees from Leaves (Poster) | |
Differentiable sorting for censored time-to-event data (Poster) | |
Data Models for Dataset Drift Controls in Machine Learning With Optical Images (Poster) | |
Differentiable Causal Discovery with Smooth Acyclic Orientations (Poster) | |