TSQueryBench: LLM-as-a-Judge for Time-Series Explanations

Natural language explanations of time-series data are increasingly produced by foundation models in high-stakes domains, making factual correctness critical. Evaluating such explanations differs fundamentally from standard natural language generation: correctness requires verifying numerical claims against structured data rather than similarity to reference text. While LLM-as-a-Judge has emerged as a scalable paradigm for text evaluation, its applicability to numerically grounded time-series explanations remains unstudied. We introduce TSQueryBench, a controlled synthetic benchmark of 500 time-series instances across 10 query types, each paired with correct, partially correct, and incorrect explanations. We evaluate six large language models across four tasks: explanation generation, relative ranking, independent scoring, and multi-anomaly detection. Our central finding is a consistent generation–evaluation asymmetry: models that fail to generate numerically correct explanations nonetheless reliably identify or score correct ones. These results show that rubric-guided LLM evaluation is substantially more reliable than generation for time-series reasoning, supporting LLM judges as scalable evaluators in numerically grounded settings. Code and data: https://anonymous.4open.science/r/TSQueryBench/.