From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework

Retrieval-augmented generation (RAG) systems suffer from a fundamental functional misalignment where retrievers optimize for semantic relevance, often recalling documents with high background utility but factually erroneous answer spans that generators blindly adopt as cognitive shortcuts. To resolve this, we propose the collaborative Critic-Reasoner framework that shifts robustness control from coarse-grained filtering to fine-grained cognitive decoupling. We disentangle the generation process into two serialized roles by deploying a Critic to perform surgical evidence purification through identifying and masking misleading entities while preserving supportive background context, followed by a Reasoner that switches from rote extraction to deductive reasoning based on the residual evidence. We operationalize this framework via a two-stage alignment strategy combining supervised fine-tuning (SFT) with path-aware direct preference optimization (DPO) to enforce strict behavioral synergy. Experimental results on adversarial benchmarks such as ConFiQA demonstrate that our method significantly outperforms baselines, achieving a 25.99\% accuracy gain in conflicting scenarios and effectively resolving the trust bias dilemma in real-world RAG.

Many AI systems answer questions by first searching through external documents and then using the retrieved information to generate an answer. However, retrieved documents are not always fully reliable: they may contain useful background information but also include a wrong answer. In such cases, current systems often either trust the wrong answer or discard the whole document, which can waste helpful evidence. This paper studies this problem in retrieval-augmented generation systems. We find that search components often retrieve documents because their background information is relevant, while generation components tend to copy specific answer spans even when those spans are wrong. To address this mismatch, we propose a collaborative Critic-Reasoner framework. The Critic removes irrelevant information and masks potentially misleading answer spans, while the Reasoner uses the remaining evidence to infer the answer instead of simply copying it. This approach helps AI systems make better use of imperfect documents and produce more reliable answers, especially when retrieved information contains conflicts or noise.