Hierarchical Abstract Tree for Cross-Document Retrieval Augmented Generation

Retrieval-augmented generation (RAG) enhances large language models with external knowledge, and tree-based RAG organizes documents into hierarchical indexes to support queries at multiple granularities. However, existing Tree-RAG methods designed for single-document retrieval face critical challenges in scaling to cross-document multi-hop questions: *(1) poor distribution adaptability*, where $k$-means clustering introduces noise due to rigid distribution assumptions; *(2) structural isolation*, as tree indexes lack explicit cross-document connections; and *(3) coarse abstraction*, which obscures fine-grained details. To address these limitations, we propose **$\Psi$-RAG**, a tree-RAG framework with two key components. *First*, a hierarchical abstract tree index built through an iterative "merging and collapse" process that adapts to data distributions without a priori assumption. *Second*, a multi-granular retrieval agent that intelligently interacts with the knowledge base with reorganized queries and an agent-powered hybrid retriever. $\Psi$-RAG supports diverse tasks from token-level question answering to document-level summarization. On cross-document multi-hop QA benchmarks, it outperforms RAPTOR by 25.9\% and HippoRAG 2 by 7.4\% in average F1 score.

Large language models are more useful when they can retrieve outside information, but existing tree-based retrieval systems mostly work on a single long document and often fail when an answer requires linking clues from several documents. We introduce Psi-RAG, which organizes a document collection into a tree index by repeatedly merging related text, without assuming the data have a simple shape. It also uses a retrieval agent that can restate the question and combine tree search with fine-grained keyword search, so it can recover both broad context and specific facts. This lets one system handle tasks from short factual questions to longer summaries. On benchmarks where answers must be assembled from multiple documents, Psi-RAG outperformed strong earlier tree- and graph-based RAG methods.