How High is ‘High’? Rethinking the Roles of Dimensionality in Topological Data Analysis and Manifold Learning

High-dimensionality of data is often regarded as a fundamental statistical impediment in Machine Learning and AI. The purpose of this paper is to clarify, on the contrary, when and how high-dimensionality may be beneficial. In the setting of a general random function model of data we delineate between three notions of dimensionality: *effective dimension* $p_{\mathrm{eff}}$, measuring total variability across feature directions; *correlation rank* $r$, measuring functional complexity across samples; and *latent intrinsic dimension* $d$ of manifold structure hidden in data. Via a generalized Hanson-Wright inequality, we show that increasing $p_{\mathrm{eff}}$ drives a *blessing of dimensionality* phenomenon, whereby data dot-products concentrate about their expectations. In turn, we show that, under mild continuity assumptions (ensuring that features bring additional information as dimension grows), persistence diagrams recover latent homology when $p_{\mathrm{eff}} \in \omega (\log n)$ as $n\to\infty$. Informed by our theory, we revisit the ground-breaking neuroscience discovery of toroidal structure in grid-cell activity made by Gardner et al. (2022): our findings provide the first empirical evidence that this structure is *isometric* to a flat torus model of physical space, suggesting that grid cell activity conveys a geometrically faithful representation of the real world.

In machine learning, working with data that has many features (known as high-dimensional data) is often seen as an obstacle where the hidden structure in the data either becomes obscured by the high dimensions, or is assumed to only be recoverable when the number of features vastly exceeds the number of samples. The former issue arises when additional features are not informative, i.e. they are mostly noise. This paper re-thinks the latter assumption, in the situation where each dimension does include some information about the hidden structure. We introduce a mathematical framework proving that if data contains enough meaningful variability, we can accurately recover its hidden structure without requiring the number of features to outnumber the number of samples. We applied our theory to groundbreaking neuroscience data tracking brain activity in rats, providing evidence that the brain activity produces a faithful representation of 2D physical space.