GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) grounds large language models with external evidence, but many implementations rely on pre-built indices that remain static after construction. Related queries therefore repeat similar multi-hop traversal, increasing latency and compute. Motivated by schema-based learning in cognitive neuroscience, we propose GAM-RAG, a training-free framework that accumulates retrieval experience from recurring or related queries and updates retrieval memory over time. GAM-RAG builds a lightweight, relation-free hierarchical index whose links capture potential co-occurrence rather than fixed semantic relations. During inference, successful retrieval episodes provide sentence-level feedback, updating sentence memories so evidence useful for similar reasoning types becomes easier to activate later. To balance stability and adaptability under noisy feedback, we introduce an uncertainty-aware, Kalman-inspired gain rule that jointly updates memory states and uncertainty estimates. It applies fast updates for reliable novel signals and conservative refinement for stable or noisy memories. We provide a theoretical analysis of the update dynamics, and empirically show that GAM-RAG improves average performance by 3.95\% over the strongest baseline and by 8.19\% with 5-turn memory, while reducing inference cost by 61\%.

Large language models often need to look up external information before answering questions, especially when the answer requires connecting evidence across multiple documents. However, many current retrieval systems use a fixed index: once the information structure is built, the system does not learn from previous searches. As a result, when users ask related questions, the system may repeatedly follow similar search paths, wasting time and computation. We propose GAM-RAG, a retrieval system that can gradually improve from past search experience without retraining the language model. Instead of storing only static document links, GAM-RAG keeps lightweight memories for sentences that have helped answer previous questions. When similar questions appear later, these memories guide the system toward useful evidence more quickly. To avoid learning from noisy or unreliable feedback, GAM-RAG updates its memory cautiously when evidence is uncertain and more strongly when the signal is reliable. Across several question-answering tasks, GAM-RAG improves answer accuracy while reducing inference cost. This suggests that retrieval systems can become more efficient and reliable by learning from their own successful search experience over time.