HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation

Embedding geometry plays a fundamental role in retrieval quality, yet dense retrievers for retrieval-augmented generation (RAG) remain largely confined to Euclidean space. However, natural language exhibits hierarchical structure from broad topics to specific entities that Euclidean embeddings fail to preserve, causing semantically distant documents to appear spuriously similar and increasing hallucination risk. To address these limitations, we introduce hyperbolic dense retrieval, developing two model variants in the Lorentz model of hyperbolic space: HyTE-FH, a fully hyperbolic transformer, and HyTE-H, a hybrid architecture projecting pre-trained Euclidean embeddings into hyperbolic space. To prevent representational collapse during sequence aggregation, we introduce the Outward Einstein Midpoint, a geometry-aware pooling operator that provably preserves hierarchical structure. On MTEB, HyTE-FH outperforms equivalent Euclidean baselines, while on RAGBench, HyTE-H achieves up to 29\% gains over Euclidean baselines in context relevance and answer relevance using substantially smaller models than current state-of-the-art retrievers. Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation—with over 20\% radial increase from general to specific concepts—a property absent in Euclidean embeddings, underscoring the critical role of geometric inductive bias in faithful RAG systems. The code is available at: https://github.com/Graph-and-Geometric-Learning/HypRAG

AI assistants increasingly answer questions by first searching a collection of documents and then writing an answer based on what they find, but if the search step pulls up the wrong material, the AI can produce confident-sounding falsehoods. These systems locate documents by turning text into points in a mathematical space and grabbing the nearest ones, and almost all of them use ordinary "flat" space. The trouble is that knowledge is shaped like a tree, with broad subjects branching into ever more specific ones, and flat space lacks the room to lay out a sprawling tree without cramming distant branches together, so unrelated documents end up looking deceptively similar. We instead place documents in hyperbolic space, a kind of geometry whose room expands the farther out you travel, giving branching hierarchies a natural fit. We also designed a new way of combining a document's parts so that specific, detailed passages stay near the edges rather than collapsing into a crowded center. Our models retrieved the right evidence and gave accurate answers up to 29% more often than comparable conventional systems, while being several times smaller, meaning fewer hallucinations and more trustworthy answers using less computing power.