ConvexBench: Can LLMs Recognize Convex Functions?

Convexity recognition plays a central role in many optimization, control, and learning problems. However, the ability of Large Language Models (LLMs) to identify this property in symbolic expressions remains unexamined. We introduce ConvexBench, a scalable and mechanically verifiable benchmark for testing whether LLMs can determine the convexity of a symbolic objective under deep functional composition. Experiments on frontier LLMs reveal a sharp \textit{compositional reasoning gap}: performance degrades rapidly with increasing depth, dropping from an F1-score of $1.0$ at depth $2$ to approximately $0.2$ at depth $100$. Inspection of models' reasoning traces indicates two failure modes: \textit{parsing failure} and \textit{lazy reasoning}. To address these limitations, we propose an agentic divide-and-conquer framework that (i) offloads parsing to an external tool to construct an abstract syntax tree (AST) and (ii) enforces recursive reasoning over each intermediate sub-expression with focused context. This framework reliably mitigates deep-composition failures, achieving substantial performance improvement at large depths (e.g., F1-Score $= 1.0$ at depth $100$).

Mathematicians often need to know whether a function is convex - a property, shaped like a valley, that makes optimization problems far easier to solve reliably. As AI systems begin assisting with real mathematical research, we wanted to know: can today's most advanced language models correctly identify this property when functions are built up by layering many simple pieces together, the way they arise in practice? To test this, we created ConvexBench, a tool that automatically generates such layered functions with known, verified answers and a tunable difficulty level. We discovered a striking weakness: leading models like GPT-5 answer perfectly when functions have only a few layers, but their accuracy collapses as the layers pile up, even though each individual step stays easy and the problem comfortably fits within the model's memory. The models either lose track of the structure or give up and guess. To fix this, we built a system that hands the parsing to a dedicated tool and then forces the model to reason through one layer at a time, seeing only what each step needs. This restored near-perfect accuracy even on the deepest problems, pointing toward more reliable AI for mathematics and science.