The Ground Truth about Legal Hallucinations

The general assumption underlying discussions of hallucinations, at least in technical literature, is that the generated output can be evaluated with reference to a ground truth, a verifiable set of facts or generally accepted knowledge. In such instance, hallucinations are generally synonymous with incorrect or false statements. When deploying LLMs for tasks involving the application of substantive legal knowledge, however, it is often difficult to compare the output to a ground truth and thus confidently declare that it constitutes a hallucination. In the case of many legal tasks, such as legal QA or contract drafting, there may be no single, accepted ground truth. Contrary to popular beliefs, which often associate the legal system with a collection of clear and unambiguous rules, it is often difficult to unequivocally state what the law is, especially in complex domains governed by a multitude of legal sources. The main focus of this paper is to demonstrate the need for a domain-specific approach to "hallucinations” in the legal domain or whenever LLMs are used in the performance of tasks involving substantive legal knowledge. Most of the existing literature addresses the problem in scenarios where the generated statements can be evaluated with reference to a ground truth or where a deviation from such ground truth is tolerable or even desirable. In the context of high-risk domains, as exemplified by law and legal services, traditional technical approaches are difficult to apply and may lead to an unintended obfuscation of the risks of using LLMs. The paper will establish the practical impossibility of developing methodologies to reliably measure the existence of hallucinations in those instances, where the term implies a deviation from a ground truth.