Skip to yearly menu bar Skip to main content


Workshop Fri, Jul 10, 2026 • 4:00 PM – 1:00 AM PDT ROOM 327

The 2nd Workshop on Connecting Low-rank Representations in AI: From Practice to Theory

Antonio Vergari ⋅ Grigorios Chrysos ⋅ Chao Li ⋅ Evrim Acar
[ OpenReview] 

Abstract

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles
6:00 PM
Invited Talks — Morning Block
Kijung Shin ⋅ Yuqi Gu
7:45 PM
Lunch Break
8:45 PM
Invited Talks — Afternoon Block
YooJung Choi ⋅ Evangelos Papalexakis
A Theory of Atomic Features and Four Testable Predictions
Kenny Peng ⋅ Jon Kleinberg ⋅ Nikhil Garg
EinSort: Sorting is All We Need for Tensorizing LLM
Toshiaki Koike-Akino ⋅ Jing Liu ⋅ Ye Wang
Impact of Connectivity on Laplacian Representations in Reinforcement Learning
Tommaso Giorgi ⋅ Pierriccardo Olivieri ⋅ Keyue Jiang ⋅ Laura Toni ⋅ Matteo Papini
Connecting Low-Rank Adapters and Policy Stability in GRPO Fine-Tuning
Antonin Rottman ⋅ Francesco Tonin ⋅ Yongtao Wu ⋅ Toshiaki Koike-Akino ⋅ Volkan Cevher
Reusable Low-Rank Subspaces Explain Why Cross-Modal Transfer Adapts with Tiny Updates
Alexis Roger ⋅ Prateek Humane ⋅ Zhenghan Tai ⋅ Gwen Legate ⋅ Andrei Mircea ⋅ Vasilii Feofanov ⋅ Irina Rish
Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models
Colin Samplawski ⋅ Ramneet Kaur ⋅ Manoj Acharya ⋅ Anirban Roy ⋅ Adam Cobb
Improved State Mixing in Higher-order and Block Diagonal Linear Recurrent Networks
Igor Dubinin ⋅ Antonio Orvieto ⋅ Felix Effenberger
Heterogeneous Low-Bandwidth Pre-Training of LLMs
Yazan Obeidi ⋅ Amir Sarfi ⋅ Joel Lidin ⋅ Paul Janson ⋅ Eugene Belilovsky
SAD LoRA: Spectral Alignment for Low-Rank Knowledge Distillation
Omer Tariq ⋅ Syed Muhammad Raza ⋅ Jeongbae Son
Scalable Deep Basis Kernel Gaussian Processes
Yunqin Zhu ⋅ Henry Yuchi ⋅ Yao Xie
Succeeding at Scale: Enterprise Retrieval Benchmark Construction and Index-Preserving Query Adaptation for Multi-Tenant Search
Prateek Jain ⋅ Shabari S Nair ⋅ Ritesh Goru ⋅ Prakhar Agarwal ⋅ Ajay Yadav ⋅ Yoga S Varadharajan ⋅ Constantine Caramanis
Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
Xinjue Wang ⋅ Xiuheng Wang ⋅ Yejun Zhang ⋅ Sergiy A Vorobyov ⋅ Esa Ollila ⋅ Zhi-Yong WANG
Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization
Laura Gomezjurado ⋅ Tatsuhiro Nakamori ⋅ Ganesh Talluri ⋅ Ansh Tiwari ⋅ Hideyuki Kawashima ⋅ Ioannis Mitliagkas ⋅ Guillaume Rabusseau ⋅ Hiroki Naganuma
Tensor Brain: Structured Probabilistic Modeling for Neural Population Activity with Tensor Networks
Yunyu Huang ⋅ Fujia Wu ⋅ DAIRUI CHEN ⋅ Xiaowei Gu ⋅ Sun Zhe ⋅ Chao Li
Dynamic BB-OPLoRA: Rigid Core, Flexible Border for Subspace-Aware Low-Rank Adaptation
Ali Sharifian ⋅ Julian Herold ⋅ Tara Gheshlaghi ⋅ Alexander Schug
Learnable Low-Rank Polynomial Sketch for Effective Linear Attention
Zongyue Qin ⋅ Haoran Deng ⋅ Jason Cong ⋅ Yizhou Sun
Sparse Fourier Regularization for Modular Arithmetic Models
Amogh Dalal ⋅ Adityasinh Rathod ⋅ Akshay Rangamani
Spectral Distillation: From Nonlinear Dynamics to Linear State-Space Models
Liane Galanti ⋅ Devan Shah ⋅ Shlomo Fortgang ⋅ Elad Hazan
Pushing the Limitation of High-order Interaction in Language Models Using Tensor Networks
Zhan Su ⋅ Fengran Mo ⋅ Guojun Liang ⋅ Prayag Tiwari
Less is More in Kolmogorov-Arnold Networks: The Shared-Atom Architecture
Sanjam Wadhwa ⋅ Hrideya Sharma ⋅ Shashank singh ⋅ Saurabh sharma
Integral Formulas for Vector Spherical Tensor Products
Valentin Heyraud ⋅ Zachary Weller-Davies ⋅ Jules Tilly
A Low-Rank Subspace Analysis of LLM Interventions
Angira Sharma ⋅ Christian Schroeder de Witt ⋅ Phil Torr ⋅ Anisoara Calinescu ⋅ Jialin Yu
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Shayan Talaei ⋅ Abhinav Chinta ⋅ Fnu Devvrit ⋅ Amin Karbasi ⋅ Azalia Mirhoseini ⋅ Amin Saberi
Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits
Andreas Grivas ⋅ Lorenzo Loconte ⋅ Emile van Krieken ⋅ Piotr Nawrot ⋅ Yu Zhao ⋅ Euan Wielewski ⋅ Pasquale Minervini ⋅ Edoardo Ponti ⋅ Antonio Vergari
Where to Approximate in Neurosymbolic Inference?
Samy Badreddine ⋅ Emile van Krieken ⋅ Luciano Serafini ⋅ Antonio Vergari
Multiplicative Mixture of Experts for Rank-Efficient LLM Finetuning
Francesco S Carzaniga ⋅ Michael Hersche ⋅ David Cox ⋅ Abbas Rahimi
BiLoCo: Binary Low-Rank Corrections for LLM FP4 Decode
David Jin ⋅ Beshr Islam Bouli ⋅ Tarushii Goel ⋅ Han Guo ⋅ Yoon Kim
CTR-Net: Continuous Tensor Ring Network for Off-the-Grid 5D Data Reconstruction
Qingfang Wang ⋅ Dawei Liu ⋅ Yisi Luo ⋅ Xiaokai Wang ⋅ Wenchao Chen ⋅ Deyu Meng
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Vladimir Bogachev ⋅ Vladimir Aletov ⋅ Alexander Molozhavenko ⋅ Sergei Kudriashov ⋅ Maxim Rakhuba
Dimension-free Bounds for Covariance Estimation with Tensor-Train Structure
Artsiom Patarusau ⋅ Nikita Puchkin ⋅ Maxim Rakhuba ⋅ Fedor Noskov
From Ricci Curvature to Metric Matching: A Simplified Approach to Geometric Transfer Learning
Sung Moon Ko ⋅ Jaewan Lee ⋅ Sumin Lee ⋅ Soorin Yim ⋅ Sehui Han
(MPO)²: Multivariate Polynomial Optimization based on Matrix Product Operators
Niccolo' Ciolli ⋅ Anders Vestergaard Nørskov ⋅ Michael Kastoryano ⋅ Petr Taborsky ⋅ Morten Mørup
OASIS: Online Activation Subspace Learning for Memory-Efficient Training
Sakshi Choudhary ⋅ Utkarsh Saxena ⋅ Kaushik Roy
Tucker Attention: A generalization of approximate attention mechanisms
Timon Klein ⋅ Jonas Kusch ⋅ Sebastian Sager ⋅ Stefan Schnake ⋅ Steffen Schotthöfer
WildCat: Near-Linear Attention in Theory and Practice
Tobias Schröder ⋅ Lester Mackey
Two-Sided Low-Rank SOAP for Efficient LLM Training
Fedor Romashov ⋅ Nikolay Yudin ⋅ Maxim Rakhuba
Provable Forgetting Bounds Drive Capacity Savings: Spectral Thresholding in Continual LoRA
Louis Barinka ⋅ Giulia Lanzillotta ⋅ Enis Simsar ⋅ Tanguy Dieudonné ⋅ Thomas Hofmann
Even Faster Kernel Matrix Linear Algebra via Density Estimation
Rikhav Shah ⋅ Sandeep Silwal ⋅ Haike Xu
Separability and Hierarchical Structure in Multilingual LLM Representations
Boris Marinov ⋅ Angira Sharma ⋅ Christian Schroeder de Witt ⋅ Phil Torr ⋅ Anisoara Calinescu ⋅ Jialin Yu
SYN-DIGITS: A Synthetic Control Framework for Calibrated Digital Twin Simulation
Grace J Fan ⋅ Chengpiao Huang ⋅ Tianyi Peng ⋅ Kaizheng Wang ⋅ Yuhang Wu
FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
Riccardo Zaccone ⋅ Stefanos Laskaridis ⋅ Marco Ciccone ⋅ Samuel Horváth
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
Paul Janson ⋅ Edouard Oyallon ⋅ Eugene Belilovsky
Tensorizing Engram: Sharing Latents Across N-Gram Embeddings is Beneficial in LLMs
Wuyang Zhou ⋅ Yuxuan Gu ⋅ Giorgos Iacovides ⋅ Yuning Qiu ⋅ Qibin Zhao ⋅ Danilo Mandic
Steering-Conditioned LoRA via Sparse Autoencoders
David ONeil Ferreira ⋅ Diogo Silva ⋅ Arlindo R Galvão Filho
RankMoD: Replacing Skips with Low-Rank Computation in Mixture of Depths
Maksim Vasilev ⋅ Xiuying Wei ⋅ Caglar Gulcehre
MoRE: Scaling mixture of experts with hardware-aware low-rank routing
Honam Wong ⋅ Surbhi Goel ⋅ Enric B Adserà
Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models
Duc Anh Nguyen ⋅ Tien N Luu ⋅ Tung Pham ⋅ Toan Tran
Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
Yu-Ang Lee ⋅ Ching-Yun Ko ⋅ Pin-Yu Chen ⋅ MI-YEN YEH
Exploration-Exploitation Generative Framework for Constrained Combinatorial Optimization with Tensor Trains
Sergei Iudin ⋅ Mikhail Podobrii ⋅ Dmitry Zheltkov ⋅ Stanislav Moiseev
FracQuant: Rank-Adaptive Image Super-resolution Network Quantization via Fractal Complexity Assessment
Chia-Ming Lee ⋅ Yu-Fan Lin ⋅ Wen-Hsin Tsai ⋅ Ming-Ching Chang ⋅ Xin Li ⋅ Chih-Chung Hsu
Revisiting NeRN: Optimizing Training Strategies for Weight Reconstruction
Hongjun Choi ⋅ Jay Thiagarajan ⋅ Ruben Glatt ⋅ Shusen Liu
Leveraging Low-Rank Structures for High-Dimensional Score-Based Sampling
Robert Gruhlke ⋅ Julius Berner ⋅ David Sommer ⋅ Lorenz Richter
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
Tanguy Dieudonné ⋅ Giulia Lanzillotta ⋅ Enis Simsar ⋅ Louis Barinka ⋅ Thomas Hofmann
ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers
Mahdi Heidari ⋅ Mohammad Mahdi Rahimi ⋅ Jaekyun Moon