Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) systems remain brittle under realistic retrieval noise, even when the required evidence appears in the top-$K$ results. A key reason is that retrievers and rerankers optimize solely for relevance, often selecting either trivial, answer-revealing passages or evidence that lacks the critical information required to answer the question, without considering whether the evidence is suitable for the generator. We propose \texttt{BAR-RAG}, which reframes the reranker as a boundary-aware evidence selector that targets the generator’s Goldilocks Zone—evidence that is neither trivially easy nor fundamentally unanswerable for the generator, but is challenging yet sufficient for inference and thus provides the strongest learning signal. \texttt{BAR-RAG} trains the selector with reinforcement learning using generator feedback, and adopts a two-stage pipeline that fine-tunes the generator under the induced evidence distribution to mitigate the distribution mismatch between training and inference. Experiments on knowledge-intensive question answering benchmarks show that \texttt{BAR-RAG} consistently improves end-to-end performance under noisy retrieval, achieving an average gain of 10.3\% over strong RAG and reranking baselines while substantially improving robustness.

AI chatbots that answer factual questions—like "Who founded this company?" or "When did this event happen?"—often look up relevant text from a large collection of documents before generating an answer. This setup, however, is surprisingly fragile: even when the right information is among the documents the AI retrieves, it often gives a wrong or made-up answer, especially when the documents are noisy or only partially relevant. We trace this problem to how the documents are chosen. Existing systems pick documents that look most relevant to the question, which sounds reasonable but has two hidden flaws: easy documents that nearly state the answer let the AI take shortcuts instead of actually reasoning, while incomplete documents leave it unable to answer at all. In both cases, the AI never learns to handle the messy, in-between cases it encounters in the real world. Our method, BAR-RAG, instead deliberately trains the AI on documents that are challenging but solvable—neither too easy nor impossible—so it learns to reason through difficult evidence. As a result, it answers factual questions more accurately and reliably, even when the documents it retrieves are imperfect.