Any-dimensional invariant universality

Several machine learning models are defined for inputs of any size, such as graphs with different numbers of nodes and point clouds with varying numbers of points. The universality properties of such any-dimensional models remain poorly understood, as universality is traditionally studied for models accepting inputs of a fixed size, defined on a compact subset of their domain. In sharp contrast, any-dimensional models can be viewed as sequences of functions defined on growing-sized inputs, and it is not clear in which sense they can be universal. We develop a systematic approach to establish any-dimensional universality by identifying any-dimensional functions with a unique function that takes inputs in a suitable infinite-dimensional limit space containing inputs of all finite sizes, as well as their limits. Using the symmetries of these inputs and relations between inputs of different sizes, we show that this limit space admits a natural topology with rich families of compact sets on which any-dimensional universality can be established. We illustrate our approach by showing that several existing architectures fail to be universal, and we propose simple modifications that restore universality.

Modern machine learning often deals with data that comes in different sizes. For example, social networks have different numbers of users, molecules have different numbers of atoms, and markets have different numbers of customers. In contrast, standard machine learning theory focuses on models that accept inputs of a fixed size. A cornerstone of that theory is universality: the guarantee that a neural network is expressive enough to learn any complex pattern when appropriately configured. While some models can accept inputs of different sizes, it remains unclear how expressive they are or what tasks they can reliably solve across these varying sizes. Our paper develops a mathematical theory to address this gap. Instead of studying one data size at a time, we treat inputs of all possible sizes as part of a single, unified space. This approach allows us to seamlessly compare data and determine when a model is truly expressive across all sizes, rather than just one. Our theory reveals that some popular architectures are poorly suited to varying input sizes, either because they incorrectly process differently-sized objects or because they lack sufficient expressive power. To overcome this, we provide practical fixes for models that process diverse structures, including sets, graphs, and point clouds of different sizes. Ultimately, this work provides a rigorous foundation for building machine learning methods that remain expressive and reliable, regardless of data size.