CiteGuard: Conformal False-Discovery Control for Faithful Retrieval-Augmented Generation

Large language models increasingly rely on retrieval-augmented generation (RAG) to ground responses in external corpora. Yet, even with strong retrievers, generated statements can remain unsupported, and the resulting citations are often not reliable indicators of evidence. We introduce CiteGuard, a RAG decoding layer that treats sentence-level factuality as a multiple-testing problem and combines conformal calibration with false-discovery-rate control. CiteGuard converts claim–evidence scores into p-values for the null hypothesis "unsupported" and uses BH/BY procedures to decide which claims to keep (with citations) and which to abstain on. On FEVER and Natural Questions, CiteGuard reduces the false-discovery rate among accepted claims from 28–31% (vanilla RAG) to below 10% at α=0.10, while retaining 86–92% of supported claims. This yields a user-controlled risk budget: practitioners can trade off faithfulness and coverage via α, with finite-sample guarantees under standard exchangeability assumptions.

Modern AI chatbots often answer questions by quoting sources from the web or a document collection, a technique called retrieval-augmented generation. But they still fabricate citations: on standard benchmarks, roughly one in three cited sentences has no real backing in the retrieved evidence. Such silent fabrication is especially dangerous in medicine, law, or news, where readers trust citations to be reliable. We built CiteGuard, a quality-control layer that sits on top of any such chatbot. Before showing each sentence, CiteGuard runs a statistical check inspired by methods that scientists use to control false alarms in genetic screens, deciding whether the evidence really supports the claim. A single dial lets developers fix the maximum acceptable fraction of unreliable sentences, and the system mathematically guarantees that limit. On standard benchmarks, CiteGuard cuts the share of unsupported claims from roughly 30% to about 10%, while still keeping most of the truly supported answers visible. By turning vague trust into a transparent risk budget, our work makes AI assistants safer to deploy in high-stakes settings without forcing them to simply refuse to answer.