Bandit Social Leaning Dynamics with Exploration Episodes

We study a stylized social learning dynamics where self-interested agents collectively follow a simple multi-armed bandit protocol. Each agent controls an "episode": a short sequence of consecutive decisions. Motivating applications include users repeatedly interacting with an AI, or repeatedly shopping at a marketplace. While agents are incentivized to explore within their respective episodes, we show that the aggregate exploration fails: e.g., its Bayesian regret grows linearly over time. In fact, such failure is a (very) typical case, not just a worst-case scenario. This conclusion persists even if an agent's per-episode utility is some fixed function of the per-round outcomes: e.g., $\min$ or $\max$, not just the sum. Thus, externally driven exploration is needed even when some amount of exploration happens organically.

Many online platforms rely on users learning from each other’s experiences. Examples include people repeatedly interacting with AI systems, shopping on online marketplaces, or using recommendation systems. In these settings, users naturally prefer options that already seem best, rather than trying uncertain alternatives that could help others learn. We study this phenomenon through a mathematical model based on multi-armed bandits, where each user makes a short sequence of decisions instead of only one. A natural question is whether this repeated interaction creates enough “organic” exploration for the system to learn good alternatives over time. Our main result is negative: even when users explore a little within their own sessions, the overall system still typically fails to learn efficiently. In fact, we show that these failures are not rare worst-case examples, but arise broadly across many settings and utility models. As a consequence, the collective learning process has Bayesian regret that grows linearly over time. These results suggest that externally incentivized exploration remains necessary in social learning systems, even when users already experiment on their own.