Skip to yearly menu bar Skip to main content

( events)   Timezone:  
Fri Jul 28 12:00 PM -- 08:00 PM (PDT) @ Meeting Room 310 None
Differentiable Almost Everything: Differentiable Relaxations, Algorithms, Operators, and Simulators
Felix Petersen · Marco Cuturi · Mathias Niepert · Hilde Kuehne · Michael Kagan · Willie Neiswanger · Stefano Ermon

Workshop Home Page

Gradients and derivatives are integral to machine learning, as they enable gradient-based optimization. In many real applications, however, models rest on algorithmic components that implement discrete decisions, or rely on discrete intermediate representations and structures. These discrete steps are intrinsically non-differentiable and accordingly break the flow of gradients. To use gradient-based approaches to learn the parameters of such models requires turning these non-differentiable components differentiable. This can be done with careful considerations, notably, using smoothing or relaxations to propose differentiable proxies for these components. With the advent of modular deep learning frameworks, these ideas have become more popular than ever in many fields of machine learning, generating in a short time-span a multitude of "differentiable everything", impacting topics as varied as rendering, sorting and ranking, convex optimizers, shortest-paths, dynamic programming, physics simulations, NN architecture search, top-k, graph algorithms, weakly- and self-supervised learning, and many more.

Opening Remarks (Remarks)
Invited Talk 1: Perturbed Optimizers for Learning (Invited Talk)
Invited Talk 2: Generalizing the Gumbel-Softmax with Stochastic Softmax Tricks (Invited Talk)
Coffee Break (Break)
Invited Talk 3: Differentiable Learning modulo Formal Verification (Invited Talk)
Short Poster Talks 1 (Short Poster Talks)
Poster Session 1 (Poster Session)
Lunch Break (Break)
Invited Talk 4: Blackbox Differentiation: the story so far (Invited Talk)
Invited Talk 5: On Differentiable Top-k Operators (Invited Talk)
Coffee Break (Break)
Short Poster Talks 2 (Short Poster Talks)
Invited Talk 6: Differentiable Rendering and Beyond (Invited Talk)
Closing Remarks (Remarks)
Poster Session 2 (Poster Session)
Dilated Convolution with Learnable Spacings: beyond bilinear interpolation (Poster)
Dynamic Control of Queuing Networks via Differentiable Discrete-Event Simulation (Poster)
Optimizing probability of barrier crossing with differentiable simulators (Poster)
EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search (Poster)
Towards Understanding Gradient Approximation in Equality Constrained Deep Declarative Networks (Poster)
End-to-end Differentiable Clustering with Associative Memories (Poster)
Probabilistic Task-Adaptive Graph Rewiring (Poster)
Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information (Poster)
Differentiable Forward Projector for X-ray Computed Tomography (Poster)
Distributions for Compositionally Differentiating Parametric Discontinuities (Poster)
Stochastic Gradient Bayesian Optimal Experimental Designs for Simulation Based Inference (Poster)
Differentiating Metropolis-Hastings to Optimize Intractable Densities (Poster)
JAX FDM: A differentiable solver for inverse form-finding (Poster)
DNArch: Learning Convolutional Neural Architectures by Backpropagation (Poster)
Efficient Surrogate Gradients for Training Spiking Neural Networks (Poster)
Plateau-Reduced Differentiable Path Tracing (Poster)
GeoPhy: Differentiable Phylogenetic Inference via Geometric Gradients of Tree Topologies (Poster)
Differentiable Clustering and Partial Fenchel-Young Losses (Poster)
Latent Random Steps as Relaxations of Max-Cut, Min-Cut, and More (Poster)
PDP: Parameter-free Differentiable Pruning is All You Need (Poster)
Sample-efficient learning of auditory object representations using differentiable impulse response synthesis (Poster)
Some challenges of calibrating differentiable agent-based models (Poster)
Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning (Poster)
SelMix: Selective Mixup Fine Tuning for Optimizing Non-Decomposable Metrics (Poster)
Lossless hardening with $\partial\mathbb{B}$ nets (Poster)
Learning Observation Models with Incremental Non-Differentiable Graph Optimizers in the Loop for Robotics State Estimation (Poster)
Fine-Tuning Language Models with Just Forward Passes (Poster)
Interpretable Neural-Symbolic Concept Reasoning (Poster)
A Gradient Flow Modification to Improve Learning from Differentiable Quantum Simulators (Poster)
Investigating Axis-Aligned Differentiable Trees through Neural Tangent Kernels (Poster)
A Short Review of Automatic Differentiation Pitfalls in Scientific Computing (Poster)
Differentiable Tree Operations Promote Compositional Generalization (Poster)
Lagrangian Proximal Gradient Descent for Learning Convex Optimization Models (Poster)
Koopman Constrained Policy Optimization: A Koopman operator theoretic method for differentiable optimal control in robotics (Poster)
Differentiable Set Partitioning (Poster)
Differentiable Search of Evolutionary Trees from Leaves (Poster)
Data Models for Dataset Drift Controls in Machine Learning With Optical Images (Poster)
SIMPLE: A Gradient Estimator for $k$-subset Sampling (Poster)
A Unified Approach to Count-Based Weakly-Supervised Learning (Poster)
PMaF: Deep Declarative Layers for Principal Matrix Features (Poster)
Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trick (Poster)
From Perception to Programs: Regularize, Overparameterize, and Amortize (Poster)
TaskMet: Task-Driven Metric Learning for Model Learning (Poster)
Differentiable sorting for censored time-to-event data (Poster)
Differentiable Causal Discovery with Smooth Acyclic Orientations (Poster)
Differentiable MaxSAT Message Passing (Poster)