Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping

Large reasoning models (LRMs) often generate long, seemingly coherent reasoning traces yet still produce incorrect answers, making hallucination detection challenging. Although trajectories contain useful signals, directly using trace text or vanilla hidden states for detection is brittle: traces vary in form and detectors can overfit to superficial patterns rather than answer validity. We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability. ARS generates counterfactual answers through small latent interventions, specifically, perturbing the trace-boundary embedding, and labels each perturbation by whether the resulting answer agrees with the original. It then learns representations that bring answer-agreeing states together and separate answer-disagreeing ones, exposing latent instability indicative of hallucination risk. The shaped embeddings are plug-and-play with existing embedding-based detectors and require no human annotations during training. Experiments demonstrate that ARS consistently improves detection and achieves substantial gains over strong baselines.

AI systems are increasingly asked to solve difficult problems by writing out step-by-step reasoning before giving a final answer. However, these reasoning steps can look convincing even when the final answer is wrong, making it hard for users to know when to trust the system. This paper proposes a way to better detect such unreliable answers. Our key idea is to check whether an answer is stable. After the model finishes its reasoning, we make very small changes to its internal state and see whether the final answer stays the same or changes. If small changes lead to different answers, this suggests that the original answer may be fragile and more likely to be unreliable. We use this stability signal to create better internal representations for detecting hallucinations. This approach does not require human-written correctness labels during training and can be combined with existing detection methods. Experiments show that it improves the ability to identify wrong answers from reasoning models, helping make AI systems more trustworthy.