Workshop
Duality Principles for Modern Machine Learning
Thomas Moellenhoff · Zelda Mariet · Mathieu Blondel · Khan Emtiyaz
Meeting Room 320
Sat 29 Jul, noon PDT
Duality is a pervasive and important principle in mathematics. Not only has it fascinated researchers in many different fields but it has also been used extensively in optimization, statistics, and machine-learning (ML), giving rise to powerful tools such as Fenchel duality in convex optimization, representer theorems in kernel methods and Bayesian nonparametrics, and dually-flat spaces in information geometry. Such applications have played an important role in the past, but lately we do not see much work on duality principles, especially in deep learning. For example, Lagrange duality can be useful for model explanation because it allows us to measure sensitivity of certain perturbations, but this is not yet fully exploited. This slowdown is perhaps due to a growing focus on nonconvex and nonlinear problems where duality does not seem to be directly applicable. There have not been any workshops on duality in recent years. With this workshop, we aim to revive the interest of the ML community in duality principles.The goal of the workshop is to bring together researchers working on various duality concepts from many different fields, and discuss new applications for modern machine learning, especially focusing on topics such as model understanding, explanation, and adaptation in deep learning and reinforcement learning.
Schedule
Sat 12:00 p.m. - 12:30 p.m.
|
Duality: Opening remarks
(
Opening remarks
)
>
SlidesLive Video |
🔗 |
Sat 12:30 p.m. - 12:55 p.m.
|
Ronny Bergman: Fenchel Duality Theory on Riemannian Manifolds and the Riemannian Chambolle-Pock Algorithm
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 12:55 p.m. - 1:05 p.m.
|
Coffee Break
|
🔗 |
Sat 1:05 p.m. - 1:17 p.m.
|
Jaeyeon Kim: Time-Reversed Dissipation Induces Duality Between Minimizing Gradient Norm and Function Value
(
Contributed talk
)
>
SlidesLive Video |
🔗 |
Sat 1:17 p.m. - 1:29 p.m.
|
Sina Baharlouei: RIFLE: Imputation and Robust Inference from Low Order Marginals
(
Contributed talk
)
>
SlidesLive Video |
🔗 |
Sat 1:30 p.m. - 1:42 p.m.
|
Joseph Shenouda: A Representer Theorem for Vector-Valued Neural Networks: Insights on Weight Decay Training and Widths of Deep Neural Networks
(
Contributed talk
)
>
SlidesLive Video |
🔗 |
Sat 1:45 p.m. - 2:10 p.m.
|
Jia-Jie Zhu: Duality from Gradient Flow Force-Balance to Distributionally Robust Learning
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 2:10 p.m. - 2:35 p.m.
|
Taiji Suzuki: Convergence of mean field Langevin dynamics: Duality viewpoint and neural network optimization
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 2:35 p.m. - 3:00 p.m.
|
Len Spek: Duality for Neural Networks through Reproducing Kernel Banach Spaces
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 2:55 p.m. - 4:00 p.m.
|
Lunch Break
|
🔗 |
Sat 4:00 p.m. - 5:30 p.m.
|
Poster session
(
Poster session
)
>
|
🔗 |
Sat 5:30 p.m. - 5:55 p.m.
|
Amy Zhang: Dual RL: Unification and New Methods for Reinforcement and Imitation Learning
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 5:55 p.m. - 6:35 p.m.
|
Coffee Break / Poster session
|
🔗 |
Sat 6:35 p.m. - 7:00 p.m.
|
Ehsan Amid: A Dualistic View of Activations in Deep Neural Networks
(
Invited talk
)
>
SlidesLive Video |
🔗 |
Sat 7:00 p.m. - 8:00 p.m.
|
Panel discussion
(
Panel discussion
)
>
SlidesLive Video |
🔗 |
-
|
Sparse Function-space Representation of Neural Networks
(
Poster
)
>
|
Aidan Scannell · Riccardo Mereu · Paul Chang · Ella Tamir · Joni Pajarinen · Arno Solin 🔗 |
-
|
Memory Maps to Understand Models
(
Poster
)
>
|
Dharmesh Tailor · Paul Chang · Siddharth Swaroop · Eric Nalisnick · Arno Solin · Khan Emtiyaz 🔗 |
-
|
Implicit Interpretation of Importance Weight Aware Updates
(
Poster
)
>
|
Keyi Chen · Francesco Orabona 🔗 |
-
|
Learning with Primal-Dual Spectral Risk Measures: a Fast Incremental Algorithm
(
Poster
)
>
|
Ronak Mehta · Vincent Roulet · Krishna Pillutla · Zaid Harchaoui 🔗 |
-
|
Estimating Joint interventional distributions from marginal interventional data
(
Poster
)
>
|
Sergio Garrido Mejia · Elke Kirschbaum · Armin Kekić · Atalanti Mastakouri 🔗 |
-
|
RIFLE: Imputation and Robust Inference from Low Order Marginals
(
Poster
)
>
|
Sina Baharlouei · Kelechi Ogudu · Peng Dai · Sze-Chuan Suen · Meisam Razaviyayn 🔗 |
-
|
A Representer Theorem for Vector-Valued Neural Networks: Insights on Weight Decay Training and Widths of Deep Neural Networks
(
Poster
)
>
|
Joseph Shenouda · Rahul Parhi · Kangwook Lee · Robert Nowak 🔗 |
-
|
Kernel Mirror Prox and RKHS Gradient Flow for Mixed Functional Nash Equilibrium
(
Poster
)
>
|
Pavel Dvurechenskii · Jia-Jie Zhu 🔗 |
-
|
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum
(
Poster
)
>
|
Amnon Geifman · Daniel Barzilai · Ronen Basri · Meirav Galun 🔗 |
-
|
Energy-Based Non-Negative Tensor Factorization via Multi-Body Modeling
(
Poster
)
>
|
Kazu Ghalamkari · Mahito Sugiyama 🔗 |
-
|
A Dual Formulation for Probabilistic Principal Component Analysis
(
Poster
)
>
|
Henri De Plaen · Johan Suykens 🔗 |
-
|
Duality in Multi-View Restricted Kernel Machines
(
Poster
)
>
|
Sonny Achten · Arun Pandey · Hannes De Meulemeester · Bart Moor · Johan Suykens 🔗 |
-
|
On the Fisher-Rao Gradient of the Evidence Lower Bound
(
Poster
)
>
|
Jesse van Oostrum · Nihat Ay 🔗 |
-
|
Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning
(
Poster
)
>
|
Yanis Bahroun · Dmitri Chklovskii · Anirvan Sengupta 🔗 |
-
|
A max-affine spline approximation of neural networks using the Legendre transform of a convex-concave representation
(
Poster
)
>
|
Adam Perrett · Danny Wood · Gavin Brown 🔗 |
-
|
Reward-Based Reinforcement Learning with Risk Constraints
(
Poster
)
>
|
Jane Lee · Konstantinos Nikolakakis · Dionysios Kalogerias · Amin Karbasi 🔗 |
-
|
The Power of Duality Principle in Offline Average-Reward Reinforcement Learning
(
Poster
)
>
|
Asuman Ozdaglar · Sarath Pattathil · Jiawei Zhang · Kaiqing Zhang 🔗 |
-
|
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
(
Poster
)
>
|
Sharan Vaswani · Amirreza Kazemi · Reza Babanezhad · Nicolas Le Roux 🔗 |
-
|
Time-Reversed Dissipation Induces Duality Between Minimizing Gradient Norm and Function Value
(
Poster
)
>
|
Jaeyeon Kim · Asuman Ozdaglar · Chanwoo Park · Ernest Ryu 🔗 |