Closing the Sim-to-Real Gap in Non-Markovian Spreading Processes via GPU-Accelerated Distributional RL

Controlling spreading processes on networks, such as epidemics, information cascades, and product adoption, requires policies that perform on realistic stochastic dynamics, not just tractable approximations. Yet policies trained on standard simplifications (mean-field ODEs, Markovian dynamics) suffer severe performance degradation at deployment. We trace this sim-to-real gap to three theoretical pathologies: Optimism Bias, where deterministic approximations systematically underestimate variance via Jensen's inequality; Hub Blindness, where global state aggregation obscures the super-spreaders driving scale-free networks; and the Valley of Death, where mean-value critics fail to navigate the bimodal nature (extinction vs. viral) of cascade outcomes. We resolve these challenges through two synergistic contributions. First, the Stratified Mean-Field Observer partitions nodes by influence tier, preserving hub dynamics at $O(N)$ cost while producing fixed-dimensional observations that enable zero-shot transfer across network scales and topologies. Second, we show that distributional RL via Truncated Quantile Critics improves risk-aware control of bimodal cascades. Trained on a GPU-accelerated simulator supporting non-Markovian renewal dynamics, our approach achieves $59\times$ improvement over Markovian baselines and robust zero-shot transfer to real-world social networks (Facebook, Twitter, YouTube), significantly mitigating the simulation-to-reality gap.

Diseases, online information, and product adoption all spread through networks, and policymakers need tools to plan interventions. The challenge: AI agents trained in standard simulations transfer poorly to real networks, where spreading is stochastic, depends on how long a person has been considering an idea (not just their current state), and is dominated by a few highly connected "hub" individuals. We develop a reinforcement learning approach that addresses all three issues. First, instead of compressing the network into a single average, we group nodes into tiers by connectivity and track each tier separately — preserving hub signals that single-number summaries destroy. Second, we build a GPU-accelerated simulator that supports realistic memory-dependent ("non-Markovian") dynamics, making large-scale training feasible in hours rather than weeks. Third, we use a risk-aware learning algorithm that treats cascade outcomes as bimodal (either fizzling out or going viral) rather than optimizing for an average outcome that rarely occurs. A policy trained once on synthetic networks transfers zero-shot to real social networks (Facebook, Twitter, YouTube) and substantially outperforms standard baselines.