Population-Aware Imitation Learning in Mean-field Games with Common Noise

Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stochasticity compels agents to adopt population-aware policies to respond to aggregate shocks. We formulate two distinct learning objectives: recovering a Nash equilibrium and maximizing performance against an expert population. We investigate two imitation proxies: Behavioral Cloning (BC) and Adversarial (ADV) divergence. We then establish finite-sample error bounds showing that minimizing these proxies effectively controls both the policy’s exploitability and its performance gap relative to the expert. Furthermore, we propose a numerical framework using generalized Fictitious Play and Deep Learning to compute expert population-aware policies. Through experiments on three environments we demonstrate that standard population-unaware policies fail to capture the equilibrium dynamics. Our results highlight that learning population-aware policies is crucial to avoid being misled by the randomness inherent in common noise.

Large crowds of people, whether in financial markets, urban traffic, or social gatherings, often behave in coordinated ways driven by shared external events: a political announcement, a weather shock, a viral trend. Understanding and predicting such collective behavior is a central challenge. A key difficulty is that these global disruptions, called common noise, force individuals to continuously adapt their strategies based on what the whole population is doing, not just their own situation. We study how an agent can learn to behave like an expert in such environments, without knowing the underlying reward structure. We address two distinct goals: i) equilibrium recovery: learning a strategy that is stable, meaning no individual has an incentive to deviate, ii) performance matching: learning a strategy that achieves rewards comparable to the expert. We extend established theoretical guarantees for imitation learning by generalizing two standard metrics, Behavioral Cloning and Adversarial divergence, to the setting with common noise, and prove that minimizing these metrics controls both goals simultaneously. We then introduce a Deep Fictitious Play heuristic to compute Nash equilibria when common noise is present, a problem for which few practical methods exist. Building on this, we propose an interactive imitation learning algorithm that trains agents to react to population-level fluctuations. Experiments consistently show that ignoring the population leads to significantly worse performance.