The DECK Taxonomy: A Universal Blind Spot in LLM Uncertainty Quantification

Uncertainty Quantification (UQ) for LLMs relies on three scorer families—black-box consistency, white-box token-probability, and LLM-as-a-Judge—whose ensembles outperform any single scorer, but why remains unexplained. In practice, most teams default to a Judge alone, with no principled basis for knowing when to add the costlier black-box or white-box families. We introduce the DECK taxonomy: a 2×2 partition of hallucinations along inter-sample consistency and token-level confidence into four regimes (Drift, Entrenched, Confabulation, Knotted), each detectable by a different combination of scorer families. Disagreement-decomposition analysis across three models and four datasets validates the taxonomy and yields a profile-driven ensemble-weighting recipe. We operationalize it as a pre-deployment diagnostic: pairwise complementarity on a small validation set reveals whether a Judge-only scorer leaves blind spots, while periodic DECK-profile re-estimation provides label-free drift monitoring in production. We further identify a universal blind spot—knowledge-gap questions on which all three families collapse to near-random AUROC—marking the operational boundary of output-level UQ and motivating abstention-aware routing.