The data manifold under the microscope

A significant gap exists between theory and practice in deep learning. Generalization and approximation error bounds are often derived for simplified models or are too loose to be informative. Many rely on the manifold hypothesis and on geometric regularity such as intrinsic dimension, curvature, and reach. Progress requires insight into data-manifold geometry and suitable benchmarks, yet existing options are polarized: analytic manifolds with known geometry but limited applicability, or real-world datasets where geometry is only coarsely estimable. We introduce a benchmarking framework for studying data geometry by repurposing and extending dSprites and COIL-20 with additional transformation dimensions and denser sampling, enabling accurate finite-difference estimates of curvature, reach, and volume that are otherwise difficult to estimate reliably and implement in practice. As applications, we assess bounds by Genovese et al. and Fefferman et al., and analyze how $\beta$-VAE layers distort, and separate known data manifolds. The results highlight the behavior of current bounds and the value of controlled benchmarks for studying internal representations. Code to reproduce the framework and experiments will be released as an open-source library upon publication. A preliminary, anonymous version is available at Zenodo for the submission.