AutoNumerics-Zero: Automated Discovery of State-of-the-Art Mathematical Functions

Transcendental functions, such as the exponential, are central to scientific computing, yet they cannot be natively calculated by digital hardware. Instead, computers must approximate these functions by combining basic operations, such as $\{+, -, \times, \div\}$, using methods like Taylor series. These methods were developed over centuries by mathematicians, who focused on approaches that could attain arbitrary accuracy. However, computers can handle most applications by using only finite-precision types, like *float32*, where any accuracy beyond the type's precision is effectively discarded. We explore, therefore, whether forgoing arbitrary accuracy can lead to the discovery of more efficient approximations. The evolutionary method of symbolic regression is particularly suitable, as it can search for arbitrary operation combinations and can optimize non-differentiable objectives, such as the number of operations used. Our results show that evolution can discover computer programs that outperform established methods in this setting, despite having no prior mathematical knowledge beyond the calculation of the basic operations. Starting from empty code, symbolic regression constructs programs representing novel mathematical expressions. In particular, we discovered a 10-operation program that approximates the exponential function to 14 significant figures, exceeding the accuracy of previously known approximations of this size by more than 6 orders of magnitude.

Essential mathematical functions, like exponentials, are fundamental to scientific simulations, but computers cannot calculate them directly. Instead, computers have to approximate these functions using basic arithmetic operations like addition and multiplication. Traditionally, these approximations were designed by mathematicians to achieve arbitrary accuracy; the higher the accuracy, the higher the cost of the calculation. Modern computer processors, however, use finite-precision data types, which means that any accuracy beyond the processor's limit is typically wasted. We hypothesized that optimizing specifically for finite-precision limits could yield much more efficient programs. We developed AutoNumerics-Zero, an evolutionary algorithm that starts with empty code and searches for computer programs from scratch. Without prior mathematical knowledge, our method constructs novel, highly efficient code. In particular, it discovered a ten-operation program that calculates exponentials to 14 significant figures. This surpasses the accuracy of previously known approximations of this size by more than a million times. Our work indicates that tailoring mathematical approximations to the constraints of modern computing can unlock significant, previously overlooked efficiency gains.