ANTiC: Adaptive Neural Temporal In Situ Compressor

The persistent storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs) have reached the petabyte-to-exabyte scale. Transient simulations modeling Navier-Stokes equations, magnetohydrodynamics, plasma physics, or binary black hole mergers generate data volumes that are prohibitive for modern high-performance computing (HPC) infrastructures. To address this bottleneck, we introduce ANTIC (Adaptive Neural Temporal in situ Compressor), an end-to-end in situ compression pipeline. ANTIC consists of an adaptive temporal selector tailored to high-dimensional physics that identifies and filters informative snapshots at simulation time, combined with a spatial neural compression module based on continual fine-tuning that learns residual updates between adjacent snapshots using neural fields. By operating in a single streaming pass, ANTIC enables a combined compression of temporal and spatial components and effectively alleviates the need for explicit on-disk storage of entire time-evolved trajectories. Experimental results demonstrate that ANTIC achieves storage reductions of approximately $\sim 400\times$ for 2D Kolmogorov flow simulations and $\sim 7000\times$ for large-scale physics simulations such as binary black hole mergers.

Modern high-fidelity physics simulations such as turbulent fluid flows, black hole mergers, or plasma gyrokinetic modeling generate data volumes at a rate that can fill entire data centers, reaching petabytes or exabytes for a single simulation run. Current high-performance computing infrastructure simply cannot store everything scientists want to keep. Thus, we introduce ANTIC, an end-to-end compression pipeline that solves this by working during the simulation itself, before data ever touches a disk. First, an adaptive temporal selector watches the evolving simulation in real time and discards time steps that carry little new physical information. For the snapshots that do matter, a neural compression module learns only what changed since the last saved frame using Continual Fine Tuning (CFT), encoding spatial structure as highly compact neural representations rather than raw numerical arrays. The entire pipeline operates in a single streaming pass, making it practical to deploy on supercomputers. In experiments across diverse PDE systems, ANTIC achieves compression factors of several orders of magnitude while faithfully preserving the physical accuracy scientists care about. This means researchers can now run higher-resolution simulations, retain more of their data, and spend less time managing storage logistics. ANTIC removes a fundamental bottleneck standing between today's computational science and the next generation of physics discovery.