LEGEX: An Eight-Jurisdiction Benchmark for Legal Review-Table Extraction
Abstract
Legal review-table systems extract structured data from judgments at scale. Their reliability across jurisdictions remains largely unmeasured. We introduce LEGEX, an expert-coded benchmark for information extraction from civil judgments. The first release covers nineteen jurisdictions and 1,320 expert-coded civil judgments. Eight core jurisdictions contain at least 100 judgments each, and eleven preview jurisdictions probe schema portability. A shared eleven-variable schema covers monetary outcomes, cost allocation, party structure, and industry classification. We evaluate one commercial review-table product (Harvey Review Tables) and two schema-constrained LLM pipelines. We report recall on expert-filled cells and false-fill rates on expert-empty cells. All three systems recover only around half of the expert-filled values. False-fill rates differ by nearly a factor of two across systems. These numbers suggest substantial headroom for improvement. LEGEX provides the labels, the schema, and the evaluation harness to measure it across jurisdictions. The code and datasets are available on https://huggingface.co/datasets/legexbenchmark.