When Condition Trees Help LLMs Apply Legal Rules: Three-Model Evidence from LegalBench
Abstract
Large language models can recall legal rules but consistently misapply them on multi-condition fact patterns. We characterize this failure as condition-evaluation shortcutting: the model produces a conclusion before all necessary conditions are independently verified. We propose Structured Rule Application (SRA), which encodes each doctrine as a condition tree of binary checks with a hard-coded boolean aggregator, evaluates each condition in an isolated LLM call, and combines them deterministically. We evaluate SRA against five baselines (zero-shot, few-shot, chain-of-thought, self-consistency, dynamic-extraction SRA, and FOL grounding) on 14 LegalBench rule-application tasks (n=2,602) across three models (GPT-4o, GPT-4o-mini, GPT-5.5). SRA improves overall accuracy by +11.0 pp on GPT-4o (87.3% vs. 76.4%) and +22.2 pp on GPT-5.5 (88.6% vs. 66.4%), but yields no net gain on GPT-4o-mini (-0.5 pp). The cross-model pattern is not monotonic in model size: GPT-5.5's baseline is worse than GPT-4o's on diversity-jurisdiction tasks (66% vs. 76%), yet SRA closes the gap to at least 91% on every variant. A controlled ablation against runtime decomposition shows that expert-encoded condition trees matter on tasks requiring precise categorical distinctions, while runtime decomposition is sufficient for clean binary rules. FOL grounding underperforms natural-language condition evaluation by 9-34 pp on SARA, because qualitative statutory conditions resist typed predicate encoding. We synthesize these findings into three exclusion criteria that identify when SRA should not be adopted, covering every observed regression in our data.