High Dimensional Learning Dynamics: the Science of Scaling

Scaling laws -- precise power-law relationships between model performance and resources (parameters, data, compute) -- have become the central organizing principle of modern large-model training. Yet the theoretical foundations of scaling remain incomplete: despite rapid recent progress, the community still lacks a unified mathematical framework connecting optimizer dynamics, architecture choice, and data structure to the observed power-law exponents that govern training at scale. This year’s HiLD workshop focuses on building a rigorous science of scaling by bringing together theoreticians and practitioners who build and train frontier models.