With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots

Reliable retrieval-augmented generation (RAG) systems depend fundamentally on the retriever’s ability to find relevant information. We show that neural retrievers used in RAG systems have blind spots, which we define as the failure to retrieve entities that are relevant to the query, but have low similarity to the query embedding. We investigate the training-induced biases that cause such blind-spot entities to be mapped to inaccessible parts of the embedding space, resulting in low retrievability. Using a large-scale dataset constructed from Wikidata relations and first paragraphs of Wikipedia, and our proposed Retrieval Probability Score (RPS), we show that blind spot risk in standard retrievers (e.g., Contriever, ReasonIR) can be predicted pre-index from entity embedding geometry, avoiding expensive retrieval evaluations. To address these blind spots, we introduce ARGUS, a pipeline that enables the retrievability of high-risk (low-RPS) entities through targeted document augmentation from a knowledge base (KB), first paragraphs of Wikipedia, in our case. Extensive experiments on BRIGHT, ImpliRet, and RAR-b show that ARGUS achieves consistent improvements across all evaluated retrievers (averaging +3.4 nDCG@5 and +4.5 nDCG@10 absolute points), with substantially larger gains in challenging subsets. These results establish that preemptively remedying blind spots is critical for building robust and trustworthy RAG systems (Code and data: github.com/ZeinabTaghavi/WithArgusEyes)

Modern AI assistants often rely on retrieval systems to find relevant information before generating an answer. However, these systems can systematically fail to retrieve certain relevant pieces of information, even when the information exists in the database. In this work, we study these failures, which we call “retrieval blind spots.” We show that these blind spots are not random. Instead, they are linked to how information is organized inside the AI model’s embedding space, making some entities consistently harder to retrieve. We introduce a new score for measuring this risk and show that blind spots can be predicted before deployment without expensive testing. To address this problem, we propose ARGUS, a method that automatically augments documents with additional contextual information for entities that are likely to be missed. Across multiple benchmarks and retrieval models, ARGUS consistently improves retrieval quality, especially in difficult cases. Our findings suggest that identifying and remedying retrieval blind spots is important for building more reliable and trustworthy AI systems.