Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

Retrieval-Augmented Generation (RAG) systems deployed in sensitive domains must provide interpretable evidence selection and robust safeguards against data poisoning, yet current approaches rely on opaque similarity-based retrieval with arbitrary top-k cutoffs that offer no explanation for their selections and remain vulnerable to adversarial manipulation. We propose METEORA, a rationale-driven RAG framework that addresses these fundamental limitations through interpretable, adaptive evidence retrieval. Our framework introduces three synergistic contributions. First, we preference-tune a general-purpose LLM to generate explicit rationales that articulate why specific evidence is needed for a given query. These rationales then guide adaptive evidence selection through a two-step process: rationale-chunk pairing for query-specific relevance assessment, followed by dynamic cutoff detection that eliminates the need for arbitrary k heuristics. Finally, the same rationales enable a verification stage that filters poisoned or misleading evidence before generation. Evaluation across six datasets demonstrates substantial improvements on three critical dimensions. For retrieval quality, METEORA achieves **21.05\%** higher precision than the best-performing baseline, while its variant with context expansion achieves **13.41\%** higher recall. In terms of efficiency, the framework reduces the volume of evidence required to reach comparable recall by **80\%**, which directly translates to a **33.34\%** improvement in downstream answer generation accuracy. Most notably for adversarial robustness, METEORA increases the F1 score from **0.10 to 0.44** under poisoning attacks, a 4.4$\times$ improvement that makes RAG systems substantially more resilient to adversarial manipulation. Human evaluation with four experienced annotators confirms genuine interpretability, achieving a mean confidence score of **3.64/5** and demonstrating that humans can reliably reconstruct evidence-level decisions with **86\% accuracy**. These results demonstrate that rationale-driven retrieval can simultaneously enhance interpretability, efficiency, and safety in RAG systems for sensitive domains. The code is available in the anonymous GitHub repository \url{https://anonymous.4open.science/r/METEORA-DC46/README.md}

Retrieval-augmented generation, or RAG, helps language models answer questions by first retrieving relevant information from external documents. Many companies now use RAG to build AI systems for law, medicine, finance, education, and enterprise search. But most RAG pipelines still depend on hidden ranking mechanisms that decide which pieces of evidence are passed to the model and which are discarded. These rankings are often difficult to inspect, can discard important evidence before the model ever sees it, and may succumb to data poisoning when misleading or adversarial content is ranked highly. METEORA addresses this problem by replacing blind ranking with rationale-driven evidence selection. Rather than simply choosing the highest-scoring chunks, METEORA evaluates whether each piece of retrieved evidence is useful for answering the query and asks for a reason why. This makes retrieval more transparent: the system does not just select evidence, it explains the basis for selecting it. The result is a more interpretable and efficient RAG pipeline. METEORA can reduce unnecessary context, preserve stronger evidence, and make the retrieval stage easier to audit. This is especially important in high-stakes settings, where users need to know not only what an AI system answered but also which evidence shaped that answer. By moving from score-based ranking to reasoned evidence selection, METEORA reframes a core problem in RAG: better retrieval is not only about finding more documents but about choosing the right evidence for the right reason.