Learning Long Timescale in Molecular Dynamics by Nano-GPT

Long-term dynamics in biomolecular processes are crucial for understanding the key evolutionary transformations of these systems. However, these long-term events requires extended simulation timescales to appear, often beyond the feasible forecast length of typical models. Consequently, the task is left to shorter but less accurate simulations. Although these simulations are brief, they are initiated with distinct perturbations, allowing them to sample the entire phase space and capture a wide range of behaviors over time. Recently, language models have been employed to learn key long-term dynamics from short simulations. However, existing approaches are limited to systems with low-dimensional reaction coordinates, projecting dynamics with memory effects. Here, we introduce nano-GPT, a novel deep learning model inspired by GPT architecture, specifically designed to manage complex dynamics and long-term dependencies in high-dimensional systems. The model employs a two-pass training structure to gradually replace MD tokens with model comprehension, thereby addressing biases in short simulations. Our findings demonstrate nano-GPT's superior ability to capture intricate dynamical properties and statistical features across extensive timescales, highlighting its potential to advance the understanding of biomolecular processes.