Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs

Symbolic regression (SR) discovers compact mathematical expressions from data, yet recent LLM-based evolutionary methods remain sample-inefficient because they rely mainly on scalar feedback such as MSE. We identify a core limitation: existing methods conflate candidate proposal with search guidance, requiring the LLM to infer how to evolve an expression, diagnose its errors, and reuse past experience from a single score. To address this, we propose Deliberate Evolution (DE), an agentic framework that decouples symbolic generation from search control. DE guides LLM proposals with adaptive operators for search direction, analytical tools for structural diagnosis, and reflective memory for trajectory-level experience. Experiments on LLM-SRBench show that DE consistently outperforms representative LLM-based SR baselines across diverse scientific domains while using only 40% of the standard sample budget. Code is available at https://github.com/Xinyu-Pang/Deliberate-Evolution.

Our paper studies how AI can discover mathematical equations from data. Existing AI-based methods often judge each candidate equation only by a single fitting score, which says whether the equation is good but not why it fails or how to improve it. We propose Deliberate Evolution, which gives the AI more useful guidance during the search. It suggests what type of change to try, checks error patterns, verifies physical consistency, and remembers lessons from previous attempts. This helps the AI revise equations more deliberately instead of relying on trial and error. Experiments show that our method finds more accurate equations with fewer search attempts.