Experience-Guided Behavior Adaptation for Large Language Models

Large language models (LLMs) cannot accumulate experience across interactions without parameter updates. Retrieval-augmented generation and memory-based approaches attempt to leverage past interactions but either rely on semantic similarity alone or ignore whether experiences actually improve performance. We introduce an uncertainty-aware guidance framework that distills compact guidance from past failures and selects it via a contextual bandit formulation. Each guidance item maintains a Beta posterior over effectiveness, and Thompson sampling balances exploration and exploitation, allowing the model to downweight unhelpful guidance over time. Across benchmarks, our method corrects up to 69.5% of prior errors and improves accuracy by up to 26%. Guidance distilled from a smaller model (Qwen3 4B) transfers effectively to a larger model (Haiku 4.5), demonstrating experience exchange across models in context space.