Skip to yearly menu bar Skip to main content


Workshop

2nd Workshop on Advancing Neural Network Training : Computational Efficiency, Scalability, and Resource Optimization (WANT@ICML 2024)

Julia Gusak ⋅ Jean Kossaifi ⋅ Alena Shilova ⋅ Rocco Sedona ⋅ Jan Kautz
Project Page

Abstract

Video

Chat is not available.

Schedule

Time zone: Conference local time zone
9:00 AM
Welcome speech from organizers
Julia Gusak
10:30 AM
Poster session
12:30 PM
Lunch
1:30 PM
Poster session
3:20 PM
Best Paper Awards
Jean Kossaifi
4:00 PM
Panel discussion
Adam DeConinck ⋅ Zach Mueller ⋅ Bruno Raffin ⋅ Max Ryabinin ⋅ Julia Gusak
4:50 PM
Closing remarks
An Analytical Approach to Enhancing DNN Efficiency and Accuracy Using Approximate Multiplication
Salar Shakibhamedan ⋅ Anice Jahanjoo ⋅ Amin Aminifar ⋅ Nima Amirafshar ⋅ Nima TaheriNejad ⋅ Axel Jantsch
Memory and Bandwidth are All Your Need for Fully Sharded Data Parallel
J. Wang ⋅ Jan Ebert ⋅ Oleg Filatov ⋅ Stefan Kesselheim
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
Wentao Guo ⋅ Jikai Long ⋅ YIMENG ZENG ⋅ Zirui Liu ⋅ Xinyu Yang ⋅ Yide Ran ⋅ Jacob Gardner ⋅ Osbert Bastani ⋅ Chris De Sa ⋅ Xiaodong Yu ⋅ Beidi Chen ⋅ Zhaozhuo Xu
DiLoCo: Distributed Low-Communication Training of Language Models
Arthur Douillard ⋅ Qixuan Feng ⋅ Andrei Rusu ⋅ Rachita Chhaparia ⋅ Yani Donchev ⋅ Adhiguna Kuncoro ⋅ Marc'Aurelio Ranzato ⋅ Arthur Szlam ⋅ Jiajun Shen
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
Zhengqing Yuan ⋅ Zhaoxu Li ⋅ Weiran Huang ⋅ Yanfang Ye ⋅ Lichao Sun
Multi-objective Differentiable Neural Architecture Search
Rhea Sukthanker ⋅ Arber Zela ⋅ Benedikt Staffler ⋅ Samuel Dooley ⋅ Josif Grabocka ⋅ Frank Hutter
Resource-constrained Neural Architecture Search on Language Models: A Case Study
Andreas Paraskeva ⋅ Joao Reis ⋅ Suzan Verberne ⋅ Jan Rijn
Accelerating Best-of-N via Speculative Rejection
Ruiqi Zhang ⋅ Momin Haider ⋅ Ming Yin ⋅ Jiahao Qiu ⋅ Mengdi Wang ⋅ Peter Bartlett ⋅ Andrea Zanette
Adaptive Model Pruning in Federated Learning through Loss Exploration
Christian Internò ⋅ Elena Raponi ⋅ Niki van Stein ⋅ Thomas Bäck ⋅ Markus Olhofer ⋅ Yaochu Jin ⋅ CITEC Barbara Hammer
Single Train Multi Deploy on Topology Search Spaces using Kshot-Hypernet
Jingyue Zhuge ⋅ Christian Mayr ⋅ Anand Subramoney ⋅ David Kappel
Enhancing Fine-grained Multi-modal Alignment via Adapters: A Parameter-Efficient Training Framework for Referring Image Segmentation
Zunnan Xu ⋅ Jiaqi Huang ⋅ Ting Liu ⋅ Yong Liu ⋅ Haonan Han ⋅ Kehong Yuan ⋅ Xiu Li
Liouna: Biologically Plausible Learning for Efficient Pre-Training of Transferrable Deep Models
Fady Rezk ⋅ Antreas Antoniou ⋅ Henry Gouk ⋅ Timothy Hospedales
Boolean Logic for Low-Energy Deep Learning
Van Minh NGUYEN ⋅ Cristian Ocampo ⋅ Aymen Askri ⋅ Ba-Hien Tran
MoReDrop: Dropout without Dropping
Li Jiang ⋅ Duo Li ⋅ Yichuan Ding ⋅ Xue Liu ⋅ Victor Chan
DrJAX: Scalable and Differentiable MapReduce Primitives in JAX
J K Rush ⋅ Zachary Charles ⋅ Zachary Garrett ⋅ Sean Augenstein ⋅ Nicole Mitchell
Variational Stochastic Gradient Descent for Deep Neural Networks
Haotian Chen ⋅ Anna Kuzina ⋅ Babak Esmaeili ⋅ Jakub Tomczak
Towards Efficient and Scalable Training of Differentially Private Deep Learning
Sebastian Rodriguez Beltran ⋅ Marlon Tobaben ⋅ Niki Loppi ⋅ Antti Honkela
Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
Brian Bartoldson ⋅ James Diffenderfer ⋅ Konstantinos Parasyris ⋅ Bhavya Kailkhura
DASH: Warm-Starting Neural Network Training Without Loss of Plasticity Under Stationarity
Baekrok Shin ⋅ Junsoo Oh ⋅ Hanseul Cho ⋅ Chulhee Yun
Efficient Adaptive Federated Optimization
Su Hyeong Lee ⋅ Sidharth Sharma ⋅ Manzil Zaheer ⋅ Tian Li
Fisher-aware Quantization for DETR Detectors with Critical-category Objectives
Huanrui Yang ⋅ Yafeng Huang ⋅ Zhen Dong ⋅ Denis Gudovskiy ⋅ Tomoyuki Okuno ⋅ Yohei Nakata ⋅ Yuan Du ⋅ EECS Kurt Keutzer ⋅ Shanghang Zhang
Efficient Document Ranking with Learnable Late Interactions
Himanshu Jain ⋅ Ziwei Ji ⋅ Ankit Singh Rawat ⋅ Andreas Veit ⋅ Sadeep Jayasumana ⋅ Sashank J. Reddi ⋅ Aditya Menon ⋅ Felix Xinnan Yu
Effective Layer Pruning Through Similarity Metric Perspective
Ian Pons ⋅ Bruno L. Yamamoto ⋅ Anna Reali ⋅ Artur Jordao Lima Correia
Model-Agnostic Graph Dataset Compression with the Tree Mover’s Distance
Mika Jain ⋅ Stefanie Jegelka ⋅ Ishani Karmarkar ⋅ Luana Ruiz ⋅ Ellen Vitercik
Scalify: scale propagation for efficient low-precision LLM training
Paul Balanca ⋅ Sam Hosegood ⋅ Carlo Luschi ⋅ Andrew Fitzgibbon
u-μP: The Unit-Scaled Maximal Update Parametrization
Charlie Blake ⋅ Constantin Eichenberg ⋅ Josef Dean ⋅ Lukas Balles ⋅ Luke Prince ⋅ Björn Deiseroth ⋅ Andres Felipe Cruz Salinas ⋅ Carlo Luschi ⋅ Samuel Weinbach ⋅ Douglas Orr
Resolving Discrepancies in Compute-Optimal Scaling of Language Models
Tomer Porian ⋅ Mitchell Wortsman ⋅ Jenia Jitsev ⋅ Ludwig Schmidt ⋅ Yair Carmon
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs
Ashwinee Panda ⋅ Berivan Isik ⋅ Xiangyu Qi ⋅ Sanmi Koyejo ⋅ Tsachy Weissman ⋅ Prateek Mittal
Asynchronous Local-SGD Training for Language Modeling
Bo Liu ⋅ Rachita Chhaparia ⋅ Arthur Douillard ⋅ Satyen Kale ⋅ Andrei Rusu ⋅ Jiajun Shen ⋅ Arthur Szlam ⋅ Marc'Aurelio Ranzato
AdaMeM: Memory Efficient Momentum for Adafactor
Nikhil Vyas ⋅ Depen Morwani ⋅ Sham Kakade
Can LLMs Enhance Performance Prediction for Deep Learning Models?
Karthick Panner Selvam ⋅ Phitchaya Phothilimthana ⋅ Sami Abu-El-Haija ⋅ Bryan Perozzi ⋅ Mats Brorsson
LoQT: Low Rank Adapters for Quantized Training
Sebastian Loeschcke ⋅ Mads Toftrup ⋅ Michael Kastoryano ⋅ Serge Belongie ⋅ Vésteinn Snæbjarnarson
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
Vijay Lingam ⋅ Atula Tejaswi ⋅ Aditya Vavre ⋅ Aneesh Shetty ⋅ Gautham Krishna Gudur ⋅ Joydeep Ghosh ⋅ Eunsol Choi ⋅ Alexandros Dimakis ⋅ Aleksandar Bojchevski ⋅ Sujay Sanghavi