CauchyNet: Compact and Data-Efficient Learning using Holomorphic Activation Functions

CauchyNet is a compact complex-valued network whose hidden units form products of shifted reciprocal features, motivated by Cauchy-type kernel representations. The design targets regression problems with sharp rational-like spikes and partially observed inputs, where standard real-valued networks often require large width. We prove that finite linear combinations of multivariate Cauchy kernels are dense in $C(M)$ on compact $M\subset\mathbb R^N$, and that CauchyNet can realize these kernel sums via its complex biases and output weights. In experiments, CauchyNet reaches lower error on near-singular and gap-filling benchmarks, often with substantially fewer trainable parameters in our settings; results on smooth and piecewise-affine targets are mixed and delineate the method's intended regime.

Today's most accurate AI models owe their power to billions of parameters and huge datasets. That recipe fails when devices are small (battery-powered sensors), data is scarce (a few hundred samples), or signals contain sharp peaks that small networks cannot resolve. We introduce CauchyNet, a compact single-layer network inspired by a 200-year-old result in mathematics — Cauchy's integral formula, which expresses smooth functions as sums of simple "one-over-distance" terms. Each hidden unit acts as a "pole," a focal point near where the target signal spikes, and the network only learns where to place these poles. Because each pole captures a sharp feature directly, CauchyNet needs far fewer units than ReLU networks, Transformers, or sinusoidal networks (SIREN). In experiments, CauchyNet cut prediction error up to a hundred-fold over equally sized ReLU networks on peaked targets, and was 5–8× more accurate than SIREN and N-BEATS at filling in missing data. We also prove mathematically that CauchyNet can approximate any continuous signal on a bounded region to arbitrary accuracy. The result suggests classical complex analysis still has much to offer modern deep learning — especially for low-power devices and scientific applications where data and compute are tight.