Skip to yearly menu bar Skip to main content


Workshop

Methods and Opportunities at Small Scale (MOSS)

Bingbin Liu · Enric Boix-Adserà · Elisabetta Cornacchia · Surbhi Goel · Abhishek Panigrahi · Eran Malach · Cyril Zhang · Benjamin Edelman

West Ballroom B

Sat 19 Jul, 8:30 a.m. PDT

The increasing computational demands of modern ML create a critical challenge: thorough experimentation becomes prohibitively expensive precisely when we most need to understand and steer model behavior. Small-scale experiments (<= 1 GPU) offer a powerful approach for systematic investigation, enabling both scientific understanding and practical advances. Recent work demonstrates the endless opportunities at this scale, including: diagnoses and mitigations of training pathologies; minimalistic replications of modern pipelines; elementary synthetic tasks that “stress test” architectures and motivate new designs; and discovery of intriguing phenomena.This workshop aims to highlight how methods and opportunities at small scale can unlock new insights and drive progress. The emphasis will be on advancing scientific understanding (and, optionally, its interplay with theory), without the need to improve state-of-the-art performance.

Live content is unavailable. Log in and register to view live content