Workshop
Data-centric Machine Learning Research (DMLR): Datasets for Foundation Models
Adam Mahdi · Ludwig Schmidt · Alexandros Dimakis · Rotem Dror · Georgia Gkioxari · Sang Truong · Lilith Bat-Leah · Fatimah Alzamzami · Georgios Smyrnis · Thao Nguyen · Nezihe Merve Gürel · Paolo Climaco · Luis Oala · Hailey Schoelkopf · Andrew M. Bean · Berivan Isik · Vaishaal Shankar · Mayee Chen · Achal Dave
Straus 3
Sat 27 Jul, midnight PDT
This workshop addresses the growing significance of preparing high quality datasets for the development of large-scale foundation models. With recent advancements highlighting the key role of dataset size, quality, diversity, and provenance in model performance, this workshop considers the strategies employed for enhancing data quality, including filtering, augmentation, and relabeling. The workshop draws upon the increasing interest in data-centric research. It seeks to advance understanding and methodologies for dataset composition and curation, ultimately fostering the development of more robust models capable of addressing diverse challenges across multiple domains and that can benefit the public.
Schedule
Sat 12:00 a.m. - 12:15 a.m.
|
Opening Remarks
(
Intro
)
>
SlidesLive Video |
🔗 |
Sat 12:15 a.m. - 1:00 a.m.
|
Invited Talk 1: Aditi Raghunathan
(
Invited Talk
)
>
SlidesLive Video |
Aditi Raghunathan 🔗 |
Sat 1:00 a.m. - 1:45 a.m.
|
Invited Talk 2: Stella Biderman
(
Invited Talk
)
>
SlidesLive Video |
Stella Biderman 🔗 |
Sat 1:45 a.m. - 2:00 a.m.
|
Break
|
🔗 |
Sat 2:00 a.m. - 3:00 a.m.
|
Panel
(
Panel
)
>
SlidesLive Video |
🔗 |
Sat 3:00 a.m. - 3:30 a.m.
|
General Announcements
(
Announcements
)
>
SlidesLive Video |
🔗 |
Sat 3:30 a.m. - 5:00 a.m.
|
Lunch Break
(
Lunch
)
>
|
🔗 |
Sat 5:00 a.m. - 5:45 a.m.
|
Invited Talk 3: Lucas Beyer
(
Invited Talk
)
>
SlidesLive Video |
Lucas Beyer 🔗 |
Sat 5:45 a.m. - 6:30 a.m.
|
Paper Talks
(
Invited Talk
)
>
SlidesLive Video |
🔗 |
Sat 6:30 a.m. - 7:00 a.m.
|
Coffee / Networking Break
|
🔗 |
Sat 7:00 a.m. - 7:50 a.m.
|
Poster Session
(
Poster Session
)
>
|
🔗 |
Sat 7:50 a.m. - 8:00 a.m.
|
Concluding Remarks
SlidesLive Video |
🔗 |
-
|
Natural Language to Class-level Code Generation by Iterative Tool-augmented Reasoning over Repository
(
Poster
)
>
|
Ajinkya Deshpande · Anmol Agarwal · Shashank Shet · Arun Iyer · Aditya Kanade · Ramakrishna Bairi · Suresh Iyengar 🔗 |
-
|
Dataset distillation for offline reinforcement learning
(
Poster
)
>
|
Jonathan Light · Yuanzhe Liu · ziniu hu 🔗 |
-
|
Understanding Bias in Visual Datasets
(
Poster
)
>
|
Boya Zeng · Yida Yin · Zhuang Liu 🔗 |
-
|
Weak-to-Strong Generalization Through the Data-Centric Lens
(
Poster
)
>
|
Changho Shin · John Cooper · Dyah Adila · Frederic Sala 🔗 |
-
|
DeepRod: A human-in-the-loop system for automatic rodent behavior analysis
(
Poster
)
>
|
Adrian Christoph Loy · Miha Garafolj · Heike Schauerte · Hanna Behnke · Cyrille Charnier · Philipp Schwarz · Kathrin Eschmann · Georg Rast · Thomas Wollmann 🔗 |
-
|
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
(
Poster
)
>
|
14 presentersMatthias Gerstgrasser · Rylan Schaeffer · Apratim Dey · Rafael Rafailov · Tomasz Korbak · Henry Sleight · Rajashree Agrawal · John Hughes · Dhruv Pai · Andrey Gromov · Dan Roberts · Diyi Yang · David Donoho · Sanmi Koyejo |
-
|
Brevity is the soul of wit: Pruning long files for code generation
(
Poster
)
>
|
Aaditya Singh · Yu Yang · Kushal Tirumala · Mostafa Elhoushi · Ari Morcos 🔗 |
-
|
A data-centric approach for assessing progress of Graph Neural Networks
(
Poster
)
>
|
Tianqi Zhao · Thi Ngan Dong · Alan Hanjalic · Megha Khosla 🔗 |
-
|
Major TOM: Expandable Datasets for Earth Observation
(
Poster
)
>
|
Mikolaj Czerkawski · Alistair Francis 🔗 |
-
|
Collaboratively Learning Robust Models from Noisy Decentralized Data
(
Poster
)
>
|
Haoyuan Li · Mathias Funk · Nezihe Merve Gürel · Aaqib Saeed 🔗 |
-
|
STENCIL: Submodular Mutual Information Based Weak Supervision for Cold-Start Active Learning
(
Poster
)
>
|
Nathan Beck · Adithya Iyer · Rishabh Iyer 🔗 |
-
|
The Tug-of-War Between Deepfake Generation and Detection ( Poster ) > link | Hannah Lee · Changyeon Lee · Kevin Farhat · Lin Qiu · Steve Geluso · Ah Young Kim · Oren Etzioni 🔗 |
-
|
Datasets for Time Series Foundation Models
(
Poster
)
>
|
Arjun Choudhry · Konrad Szafer · Mononito Goswami · Yifu Cai · Artur Dubrawski 🔗 |
-
|
A Comparative Analysis of Influence Signals for Data Debugging
(
Poster
)
>
|
Nikolaos Myrtakis · Ioannis Tsamardinos · VASSILIS CHRISTOPHIDES 🔗 |
-
|
AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic
(
Poster
)
>
|
Emad Alghamdi · Reem Masoud · Deema Alnuhait · Afnan Alomairi · Ahmed Ashraf · Mohamed Zaytoon 🔗 |
-
|
Large-Scale Dataset Pruning in Adversarial Training through Data Importance Extrapolation
(
Poster
)
>
|
Björn Nieth · Thomas Altstidl · Leo Schwinn · Bjoern Eskofier 🔗 |
-
|
FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models
(
Poster
)
>
|
Hanjun Luo · Ziye Deng · Ruizhe Chen · Zuozhu Liu 🔗 |
-
|
Spurious Correlations in Machine Learning: A Survey
(
Poster
)
>
|
Wenqian Ye · Guangtao Zheng · Xu Cao · Yunsheng Ma · Aidong Zhang 🔗 |
-
|
Resource Efficient Datasets for Inferring Parameters of Differential Equations
(
Poster
)
>
|
Antanas Murelis · Mojmir Mutny · Lenart Treven · Ugne Sakenyte · Andreas Krause 🔗 |
-
|
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
(
Poster
)
>
|
Rylan Schaeffer · Hailey Schoelkopf · Brando Miranda · Gabriel Mukobi · Varun Madan · Adam Ibrahim · Herbie Bradley · Stella Biderman · Sanmi Koyejo 🔗 |
-
|
Evaluating $n$-Gram Novelty of Language Models
(
Poster
)
>
|
William Merrill · Noah Smith · Yanai Elazar 🔗 |
-
|
Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark
(
Poster
)
>
|
Elizabeth M Fons Etcheverry · Rachneet Kaur · Soham Palande · Zhen Zeng · Svitlana Vyetrenko · Tucker Balch 🔗 |
-
|
BioinformaticsBench: A collaboratively built large language model benchmark for Bioinformatics reasoning
(
Poster
)
>
|
Varuni Sarwal · Seungmo Lee · Rosemary He · Aingela Kattapuram · Xiaoxuan Wang · Yijia Xiao · Serghei Mangul · Wei Wang 🔗 |
-
|
Evaluating Music Understanding in Multimodal Audio-Language Models
(
Poster
)
>
|
Benno Weck · Ilaria Manco · Emmanouil Benetos · Elio Quinton · George Fazekas · Dmitry Bogdanov 🔗 |
-
|
Understanding Hallucinations in Diffusion Models through Mode Interpolation
(
Poster
)
>
|
Sumukh K Aithal · Pratyush Maini · Zachary Lipton · Zico Kolter 🔗 |
-
|
Many Perception Tasks are Highly Redundant Functions of their Input Data
(
Poster
)
>
|
Rahul Ramesh · Anthony Bisulco · Ronald Di Tullio · Linran Wei · Vijay Balasubramanian · Kostas Daniilidis · Pratik Chaudhari 🔗 |
-
|
Truthful Dataset Valuation by Pointwise Mutual Information
(
Poster
)
>
|
SHURAN ZHENG · Yongchan Kwon · Xuan Qi · James Zou 🔗 |
-
|
On Evaluation of Vision Datasets and Models using Human Competency Frameworks
(
Poster
)
>
|
Rahul Ramachandran · Tejal Kulkarni · Charchit Sharma · Deepak Vijaykeerthy · Vineeth N Balasubramanian 🔗 |
-
|
Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
(
Poster
)
>
|
Fadhel Ayed · Soufiane Hayou 🔗 |
-
|
KPC-cF: Korean Aspect-Based Sentiment Analysis via NLI-Based Pseudo-Classifier with Corpus Filtering
(
Poster
)
>
|
Kibeom Nam 🔗 |
-
|
AdaEmbed: Semi-supervised Domain Adaptation in the Embedding Space
(
Poster
)
>
|
Ali Mottaghi · Muhammad Abdullah Jamal · Serena Yeung · Omid Mohareri 🔗 |
-
|
VideoPhy: Evaluating Physical Commonsense In Video Generation
(
Poster
)
>
|
Hritik Bansal · Zongyu Lin · Tianyi Xie · Zeshun Zong · Chenfanfu Jiang · Yizhou Sun · Kai-Wei Chang · Aditya Grover 🔗 |
-
|
Multilingual Diversity Improves Vision-Language Representations
(
Poster
)
>
|
Thao Nguyen · Matthew Wallingford · Sebastin Santy · Wei-Chiu Ma · Sewoong Oh · Ludwig Schmidt · Pang Wei Koh · Ranjay Krishna 🔗 |
-
|
Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts
(
Poster
)
>
|
Jacob Haimes · Cenny Wenner · Kunvar Thaman · Vassil Tashev · Clement Neo · Esben Kran · Jason Schreiber, né Hoelscher-Obermaier 🔗 |
-
|
Robust 6DoF Pose Estimation Against Depth Noise and a Comprehensive Evaluation on a Mobile Dataset ( Poster ) > link | Zixun Huang · Keling Yao · Seth Zhao · Chuanyu Pan · Chenfeng Xu · Kathy Zhuangs · Tianjian Xu · Weiyu Feng · Allen Yang 🔗 |
-
|
The AI Disguise: Applying Advanced Tokenization and POS Techniques for Text Authenticity
(
Poster
)
>
|
Andrew Zhang 🔗 |
-
|
Data Generation using Large Language Models for Text Classification: An Empirical Case Study
(
Poster
)
>
|
Yinheng Li · Rogerio Bonatti · Sara Abdali · Justin Wagle · Kazuhito Koishida 🔗 |
-
|
Bayesian Data Selection
(
Poster
)
>
|
Julian Rodemann 🔗 |
-
|
Towards Bridging Classical and Neural Computation through a Read-Eval-Print Loop
(
Poster
)
>
|
David Zhang · Michaël Defferrard · Corrado Rainone · Roland Memisevic 🔗 |
-
|
Automated Data Curation for Robust Language Model Fine-Tuning
(
Poster
)
>
|
Jiuhai Chen · Jonas Mueller 🔗 |
-
|
Towards Ontology-Enhanced Representation Learning for Large Language Models ( Poster ) > link | Francesco Ronzano · Jay Nanavati 🔗 |
-
|
In Search of Forgotten Domain Generalization
(
Poster
)
>
|
Prasanna Mayilvahanan · Roland S. Zimmermann · Thaddäus Wiedemer · Evgenia Rusak · Attila Juhos · Matthias Bethge · Wieland Brendel 🔗 |
-
|
Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
(
Poster
)
>
|
Hritik Bansal · Ashima Suvarna · Gantavya Bhatt · Nanyun Peng · Kai-Wei Chang · Aditya Grover 🔗 |
-
|
The CLRS-Text Algorithmic Reasoning Benchmark
(
Poster
)
>
|
Larisa Markeeva · Sean McLeish · Borja Ibarz · Wilfried Bounsi · Olga Kozlova · Alex Vitvitskyi · Charles Blundell · Tom Goldstein · Avi Schwarzschild · Petar Veličković 🔗 |
-
|
Data Mixture Inference Attack: BPE Tokenizers Reveal Training Data Compositions
(
Poster
)
>
|
Jonathan Hayase · Alisa Liu · Yejin Choi · Sewoong Oh · Noah Smith 🔗 |
-
|
Quantifying Spuriousness of Biased Datasets Using Partial Information Decomposition
(
Poster
)
>
|
Barproda Halder · Faisal Hamman · Sachindra P Dissanayake · Richard Zhang · Ilia Sucholutsky · Sanghamitra Dutta 🔗 |
-
|
Data Shapley in One Training Run
(
Poster
)
>
|
Jiachen Wang · Prateek Mittal · Dawn Song · Ruoxi Jia 🔗 |
-
|
What is the Right Notion of Distance between Predict-then-Optimize Tasks?
(
Poster
)
>
|
Paula Rodriguez-Diaz · Kai Wang · David Alvarez-Melis · Milind Tambe 🔗 |
-
|
Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions
(
Poster
)
>
|
Jingtan Wang · Xiaoqiang Lin · Rui Qiao · Chuan-Sheng Foo · Bryan Kian Hsiang Low 🔗 |
-
|
Compute-efficient LLM Training via Online Batch Selection
(
Poster
)
>
|
Jiachen Wang · Tong Wu · Dawn Song · Prateek Mittal · Ruoxi Jia 🔗 |
-
|
Does your data spark joy? Performance gains from domain upsampling at the end of training
(
Poster
)
>
|
Cody Blakeney · Mansheej Paul · Brett Larsen · Sean Owen · Jonathan Frankle 🔗 |
-
|
Is Aligned Data the Optimal Training Data?
(
Poster
)
>
|
Elyas Obbad · Brando Miranda 🔗 |
-
|
No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision–Language Models
(
Poster
)
>
|
Angéline Pouget · Lucas Beyer · Emanuele Bugliarello · Xiao Wang · Andreas Steiner · Xiaohua Zhai · Ibrahim Alabdulmohsin 🔗 |
-
|
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning
(
Poster
)
>
|
Yiping Wang · Yifang Chen · Wendan Yan · Alex Fang · Wenjing Zhou · Kevin Jamieson · Simon Du 🔗 |
-
|
The Neglected Tails in Vision-Language Models
(
Poster
)
>
|
Shubham Parashar · Zhiqiu Lin · Tian Liu · Xiangjue Dong · Yanan Li · Deva Ramanan · James Caverlee · Shu Kong 🔗 |
-
|
Efficient and Accurate Explanation Estimation with Distribution Compression
(
Poster
)
>
|
Hubert Baniecki · Giuseppe Casalicchio · Bernd Bischl · Przemyslaw Biecek 🔗 |
-
|
Stronger Than You Think: Benchmarking Weak Supervision on Realistic Tasks
(
Poster
)
>
|
Tianyi Zhang · Linrong Cai · Nicholas Roberts · Jeffrey Li · Neel Guha · Frederic Sala 🔗 |
-
|
COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Learning
(
Poster
)
>
|
Arnav M Das · Gantavya Bhatt · Lilly Kumari · Sahil Verma · Jeff Bilmes 🔗 |
-
|
Training-free Design of Augmentations with Data-centric Principles
(
Poster
)
>
|
Jieke Wu · Wei Huang · Mingyuan Bai · Xiaoling Hu · Yi Duan · Wuyang Chen 🔗 |
-
|
HyperINF: Scaling-up Accurate Approximation of Influence Functions by the Hyperpower Method
(
Poster
)
>
|
xinyu Zhou · Simin Fan · Martin Jaggi 🔗 |
-
|
Learning to Reason by Failing: Offline RL on Sub-optimal Rollouts Scales Synthetic Data by 8x
(
Poster
)
>
|
Amrith Setlur · Saurabh Garg · Xinyang Geng · Naman Garg · Virginia Smith · Aviral Kumar 🔗 |
-
|
Efficient Ensembles Improve Training Data Attribution
(
Poster
)
>
|
Junwei Deng · Ting-Wei Li · Shichang Zhang · Jiaqi Ma 🔗 |
-
|
Understanding the Gains from Repeated Self-Distillation
(
Poster
)
>
|
Divyansh Pareek · Simon Du · Sewoong Oh 🔗 |
-
|
Open Artificial Knowledge
(
Poster
)
>
|
Vadim Borisov · Richard Schreiber 🔗 |
-
|
CONTEXTUAL: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models
(
Poster
)
>
|
Rohan Wadhawan · Hritik Bansal · Kai-Wei Chang · Nanyun Peng 🔗 |
-
|
Community search signatures as foundation features for human-centered geospatial modeling
(
Poster
)
>
|
Mimi Sun · Chaitanya Kamath · MOHIT AGARWAL · Arbaaz Muslim · Yangli Yee · David Schottlander · Shailesh Bavadekar · Niv Efron · SHRAVYA SHETTY · Gautam Prasad 🔗 |
-
|
Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach
(
Poster
)
>
|
13 presentersVan Huy Vo · Vasil Khalidov · Timothée Darcet · Théo Moutakanni · Nikita Smetanin · Marc Szafraniec · Hugo Touvron · Camille Couprie · Maxime Oquab · Armand Joulin · Herve Jegou · Patrick Labatut · Piotr Bojanowski |
-
|
PabLO: Improving Semi-Supervised Learning with Pseudolabeling Optimization
(
Poster
)
>
|
Harit Vishwakarma · Yi Chen · Satya Sai Srinath Namburi GNVV · Sui Jiet Tay · Ramya Vinayak · Frederic Sala 🔗 |
-
|
Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist
(
Poster
)
>
|
Zihao Zhou · Shudong Liu · Maizhen Ning · Wei Liu · Derek Wong · Jindong Wang · Qiufeng Wang · Kaizhu Huang 🔗 |
-
|
Pearls from Pebbles: Improved Confidence Functions for Auto-labeling
(
Poster
)
>
|
Harit Vishwakarma · Yi Chen · Sui Jiet Tay · Satya Sai Srinath Namburi GNVV · Frederic Sala · Ramya Vinayak 🔗 |
-
|
What Data-Centric AI Can Do For k-means: a Faster, Robust kmeans-d
(
Poster
)
>
|
PARICHIT SHARMA · HASAN KURBAN · Mehmet Dalkilic 🔗 |