Position: Hallucinations Undermine Trust; Metacognition is a Way Forward

Despite significant improvements in factuality, confident errors continue to reappear as benchmarks probe more niche knowledge that models lack. We argue that most gains have come from expanding the model's knowledge boundary (encoding more facts) rather than improving awareness of that boundary (distinguishing known from unknown). We conjecture that this stems from the fact that the latter is inherently difficult: in the absence of strong ability to separate correct from incorrect answers (discrimination), fully eliminating hallucinations requires aggressive abstention, imposing a significant utility tax. Given this limitation, we propose complementing knowledge expansion with faithful uncertainty -- honestly conveying whatever uncertainty remains. This metacognitive capability becomes even more critical for tool-augmented models, where it serves as the control layer that determines when to search and how to weigh conflicting information. We conclude by highlighting the key challenges and open problems that must be tackled to make progress toward this objective.