Methods and Opportunities at Small Scale (MOSS)

The increasing computational demands of modern ML create a critical challenge: thorough experimentation becomes prohibitively expensive precisely when we most need to understand and steer model behavior. Small-scale experiments (<= 1 GPU) offer a powerful approach for systematic investigation, enabling both scientific understanding and practical advances. Recent work demonstrates the endless opportunities at this scale, including: diagnoses and mitigations of training pathologies; minimalistic replications of modern pipelines; elementary synthetic tasks that “stress test” architectures and motivate new designs; and discovery of intriguing phenomena.This workshop aims to highlight how methods and opportunities at small scale can unlock new insights and drive progress. The emphasis will be on advancing scientific understanding (and, optionally, its interplay with theory), without the need to improve state-of-the-art performance.