Hallucination Detection from Structural Reasoning Model

Hallucinations pose a key challenge for large language models, and chain-of-thought prompting exposes intermediate reasoning but usually treats traces as linear sequences, making crossstep dependencies and unsupported intermediate claims difficult to identify. We propose a structural reasoning model to describe interactions among local reasoning steps. To detect hallucinations, we extract a directed acyclic reasoning graph over conditions and intermediate claims, verify each claim against its parent nodes, and aggregate the step signals with a simple mass-flow rule. Under a probabilistic erasure-gate abstraction, we interpret this aggregation as measuring information loss along the reasoning graph. Experiments on GSM8K, MATH, HumanEval, and HotpotQA show that the proposed method is most advantageous on longer, dependency-rich reasoning traces such as math and code generation, while remaining competitive on shorter factual QA; these results provide a structured perspective on chain-of-thought evaluation. All code and data are available at https://github.com/soncheinbok/FlowScore.

AI language models can give answers that sound convincing but are wrong, especially when solving problems that require several reasoning steps. A common way to inspect such answers is to look at the model’s step-by-step explanation, but a long list of steps can still hide where an unsupported claim first appears. This paper proposes a way to make these explanations easier to check. Instead of treating the explanation as a simple sequence, the method asks the model to state what each step depends on, checks each step only against the information it claims to use, and then combines these local checks into one warning score for the whole answer. Experiments on arithmetic, advanced math, coding, and questions that require combining several facts show that this structured checking approach finds incorrect answers better than several existing methods. The goal is to help users and developers spot unreliable AI answers earlier, while recognizing that automated checks can still make mistakes and should not replace careful human review.