CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG

Retrieval augmented generation (RAG) has enhanced large language models by enabling access to external knowledge, with graph-based RAG emerging as a powerful paradigm for structured retrieval and reasoning. However, existing graph-based methods often over-rely on entity-centric node matching and lack explicit causal modeling, leading to unfaithful or spurious answers. Prior attempts to incorporate causality are typically limited to local or single-document contexts and also suffer from information isolation that arises from modular graph structures, which hinders scalability and cross-module causal reasoning. To address these challenges, we propose CausalRAG2, a framework that rethinks knowledge organization for graph-based RAG through causal gating across hierarchical modules. CausalRAG2 explicitly models causal relationships to suppress spurious correlations while enabling scalable reasoning over large-scale knowledge graphs. We also introduce HolisQA, a benchmark for holistic comprehension beyond entity-centric matching. Extensive experiments demonstrate that CausalRAG2 consistently outperforms competitive graph-based RAG baselines across multiple datasets and evaluation metrics. Our work establishes a principled foundation for structured, scalable, and causally grounded RAG systems.

Large language models are increasingly connected to external knowledge systems so they can look up information before answering questions. Many recent systems organize knowledge as graphs, where related ideas are connected together. But as these graphs become larger, the information often breaks into separate clusters. This can cause AI systems to get stuck searching within one local region while missing important information elsewhere. They may also combine facts that appear related on the surface but are not actually connected in a meaningful way. In our work, we develop a new retrieval system called CausalRAG2 to help AI models better organize and navigate knowledge. Instead of only following semantic similarity, our method tries to identify meaningful causal connections between pieces of information. This allows the system to jump across disconnected regions of a knowledge graph and assemble more complete reasoning chains while filtering out misleading associations. We also introduce a new benchmark called HolisQA to evaluate whether AI systems can understand complex materials as a whole rather than simply matching keywords or entities. Across multiple domains, our method produces more reliable and better grounded answers than existing graph-based retrieval systems.