Zero-source LLM Hallucination Detection with Human-like Criteria Probing

Large language models (LLMs) often hallucinate by generating factually incorrect or unfaithful content, posing significant risks to their safe use. Detecting such hallucinations is particularly challenging under the zero-source constraint, where no model internals or external references are available, and detection must rely solely on the textual query–answer pair. In this paper, we propose Human-like Criteria Probing for Hallucination Detection (HCPD), a paradigm that emulates the multi-faceted reasoning of human evaluators. Its core is a Human-like Criteria Probing (HCP) mechanism, in which a LLM agent adaptively decomposes its judgment into a weighted set of interpretable criteria and aggregates criterion-specific scores into a final truthfulness measure. To achieve this adaptive capability, we introduce a reward-based alignment scheme using only weak supervision from semantic consistency. At inference, we employ a multi-sampling aggregation strategy to ensure robust decisions while preserving full interpretability. We further provide theoretical analysis supporting the reliability of our approach. Extensive experiments show that HCPD consistently outperforms state-of-the-art baselines, offering an effective and explainable solution for zero-source hallucination detection.

AI chatbots can sometimes present made-up facts as if they were true, a problem known as "hallucination". Catching these errors is especially difficult when we can only see the chatbot's text output, without access to its internal workings or any reference material to check against. This is the most common real-world scenario: imagine a social media platform needing to verify millions of AI-generated posts without knowing which AI system wrote them. We noticed that human experts evaluate text by considering multiple aspects at once (checking factual accuracy, logical reasoning, and contextual consistency) and weighting these aspects differently depending on the topic. Our approach teaches an AI agent to mimic this process: it automatically identifies the most relevant quality criteria for each response, assigns importance to each, and produces an overall trustworthiness score along with a clear explanation of its reasoning. Experiments show that our method significantly outperforms existing approaches across a range of tasks and AI systems, while also telling users exactly why a response was flagged, making AI-generated content easier to trust and verify.