Beyond Accuracy: A Dual-Judge Evaluation Protocol for Vision-Language Models in Legally Grounded Tasks

AI systems are increasingly deployed in domains where the law requires decisions to be justifiable alongside being correct, and the AI for Law community has built substantial evaluation infrastructure for this space — including LexGLUE, LegalBench, and the LLM-as-judge methodology of Zheng et al. (2023). We contribute one additional signal that complements this infrastructure: a dual-judge evaluation protocol pairing the standard 0–10 LLM-as-judge with a strict binary semantic-equivalence judge against a human-curated legal reference. On a controlled testbed of 4,680 traffic-sign evaluations under seven visibility levels and two occlusion modes, the two judges agree at Pearson r=0.644, with an asymmetric 8.0% disagreement (Type II) that varies with input difficulty and peaks at 14.2% in the clean-input regime (v=0.8). A 49-row human-eval validation (Pearson r=0.81 with the 0–10 judge; r=0.80 with the LLM accuracy sub-score) confirms that the equivalence judge applies an appropriately strict reference-match criterion. The protocol adds one LLM call per evaluation and surfaces a deployability-relevant signal that a single-judge protocol does not report; we release the prompt template, the occluded variants, and the full evaluation results for adoption by other AI-for-Law benchmarks.