On the Plasticity and Stability for Post-Training Large Language Models

Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads to destructive interference. Crucially, we argue that deterministic projection methods are suboptimal for GRPO as they overlook the intrinsic stochasticity of group-based gradient estimates. To address this, we propose Probabilistic Conflict Resolution (PCR), a Bayesian framework that models gradients as random variables. PCR dynamically arbitrates conflicts via an uncertainty-aware ``soft projection'' mechanism, optimizing the signal-to-noise ratio. Extensive experiments demonstrate that PCR significantly smooths the training trajectory and achieves superior performance in various reasoning tasks.

How do we teach LLMs complex reasoning without causing it to forget its general knowledge? A popular training method, GRPO, often becomes highly unstable because the instructions for learning new skills and retaining old ones clash, canceling each other out. To solve this, we introduce Probabilistic Conflict Resolution (PCR). Instead of using rigid rules, PCR acts as an intelligent mediator that measures the confidence and uncertainty of conflicting instructions. It dynamically blends them together, amplifying clear, reliable learning signals while filtering out chaotic noise. Extensive experiments demonstrate the effectiveness and advantages of PCR.