Beyond Accuracy and Complexity: The Effective Information Criterion for Structurally Stable Symbolic Regression

Symbolic regression (SR) traditionally balances accuracy and complexity, implicitly assuming that simpler formulas are structurally more rational. We argue that this assumption is insufficient: existing algorithms often exploit this metric to discover accurate and compact but structurally irrational formulas that are numerically ill-conditioned and physically inexplicable. Inspired by the structural stability of real physical laws, we propose the Effective Information Criterion (EIC) to quantify formula rationality. EIC models formulas as information channels and measures the amplification of inherent rounding noise during recursive calculation, effectively distinguishing physically plausible structures from pathological ones without relying on ground truth. Our analysis reveals a stark structural stability gap between human-derived equations and SR-discovered results. By integrating EIC into SR workflows, we provide explicit structural guidance: for heuristic search, EIC steers algorithms toward stable regions to yield superior Pareto frontiers; for generative models, EIC-based filtering improves pre-training sample efficiency by 2–4 times and boosts generalization $R^2$ by 22.4\%. Finally, an extensive study with 108 human experts shows that EIC aligns with human preferences in 70\% of cases, validating structural stability as a critical prerequisite for human-perceived interpretability. We release our code at https://github.com/tsinghua-fib-lab/EIC

Symbolic regression is a type of white-box machine learning: instead of only making predictions, it searches for mathematical formulas that fit observed data. These formulas can help scientists understand the hidden relationships behind their observations. Although symbolic regression can produce concise formulas that fit data well, we find that many of them have ill-conditioned structures, in which tiny numerical errors that arise during calculation, such as rounding errors, can be greatly magnified. Real physical equations, by contrast, usually remain stable under such rounding errors. This difference suggests that many machine-discovered formulas, while compact and accurate, may still be poor explanations of how variables truly relate to one another. We introduce the Effective Information Criterion (EIC), a measure that directly evaluates how structurally stable a formula is. EIC reveals a clear structural-stability gap between formulas found by current symbolic regression algorithms and real physical equations. Adding EIC to existing symbolic regression methods helps these methods find more reliable formulas, train more efficiently, and better match human experts’ intuition about interpretability. This work provides a practical tool for making equations discovered by symbolic regression more trustworthy for scientific discovery.