RAGEN-2: Reasoning Collapse in Agentic RL

RL training of multi-turn LLM agents is unstable, and reasoning quality drives task performance. Entropy, the standard reasoning-stability monitor, only measures within-input diversity and misses whether reasoning depends on the input. We identify template collapse: stable entropy alongside input-agnostic boilerplate, invisible to entropy and existing metrics. We diagnose it via a mutual-information (MI) proxy that scores cross-input distinguishability online; across tasks, MI correlates with final performance far more strongly than entropy. We then explain collapse via a signal-to-noise ratio (SNR) mechanism: low within-input reward variance weakens task gradients, letting input-agnostic regularization dominate and erase cross-input differences. We mitigate this with SNR-Aware Filtering, prioritizing high-variance prompts each iteration. Across planning, math reasoning, web navigation, and code execution, the method consistently improves input dependence and task performance.