MetaDNS: Enhancing Exploration in Discrete Neural Samplers via Metadynamics

Sampling from discrete distributions with multiple modes and energy barriers is fundamental to machine learning and computational physics. Recent discrete neural samplers like MDNS suffer from mode collapse and fail to sample high-energy barrier regions between modes, which is critical for free energy estimation and understanding phase transitions. We propose Metadynamics Discrete Neural Sampler (MetaDNS), a general framework integrating well-tempered metadynamics into discrete diffusion or autoregressive samplers. By maintaining an adaptive, history-dependent bias potential along selected low-dimensional coordinates, MetaDNS forces exploration of previously inaccessible regions, enabling free energy reconstruction infeasible with standard neural samplers due to a lack of high-energy samples. On challenging low-temperature benchmarks including Ising, Potts, and the copper-gold binary alloy, MetaDNS reproduces the thermodynamic distribution. Compared to MCMC-based metadynamics, MetaDNS also achieves comparable exploration requiring fewer bias deposition steps.

Understanding how materials behave (for example, why a copper-gold alloy forms an ordered crystal at low temperatures but becomes disordered when heated) requires exploring an astronomically large number of possible atomic arrangements. Modern ML-based methods can learn these arrangements efficiently, but they suffer from a critical flaw: they repeatedly generate the same configurations and miss entire families of arrangements that are crucial for understanding material behavior. This is like a hiker who keeps returning to the same valleys and never discovers the rest of the landscape. Our method, MetaDNS, solves this by borrowing an idea from computational chemistry called metadynamics. As the ML model generates atomic arrangements, it keeps a running memory of where it has already been and adds a repulsive "penalty" to those visited regions, like gradually filling in explored valleys so the hiker is nudged toward new terrain. This forces the model to discover configurations it would otherwise miss, including the rare high-energy arrangements that bridge stable phases and reveal how phase transitions occur. We validated MetaDNS on magnetic spin models and the copper-gold alloy, showing it successfully recovers all stable phases and reconstructs the energy landscape needed to predict thermodynamic behavior, capabilities that state-of-the-art ML samplers alone cannot achieve.