Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

Energy-based models (EBMs) are flexible generative architectures inspired by statistical physics, but their learning and generative properties remain poorly understood. We analyze a solvable EBM in the high-dimensional limit, the spherical Boltzmann machine (SBM), combining random matrix theory and dynamical mean-field theory. We solve exact equations for its training dynamics; compute the Bayesian evidence, which acts as a partition function over parameters; and uncover cascades of phase transitions, both at equilibrium and along training, driven by the alignment and condensation of the top modes of the coupling matrix. In a teacher--student setting we connect these transitions to four generative phenomena: sampling temperature tuning, double descent of generative metrics versus regularization, tempered-posterior effects in posterior-predictive ensembles, and out-of-equilibrium training dynamics that bias the converged model. Numerical experiments on Potts BMs, normalizing flows, autoregressive networks, RBMs, and Bayesian GANs reproduce the same phenomenology beyond the SBM. Code is provided in the supplementary material.