Invited Talk #1: Model-Agnostic Equivariance for Efficient Learning under Symmetry

Symmetries are ubiquitous in machine learning. While hard-coding equivariance into model architectures can be a useful inductive bias, it is often overly restrictive. This talk studies model-agnostic mechanisms for enforcing equivariance, focusing on two common variants: data augmentation, which symmetrizes the training objective through transformed data, and group averaging, which symmetrizes models or predictions by averaging over group actions. In both cases, exact symmetry requires averaging over the full group, which is often computationally infeasible for large or infinite groups. We develop a theory of the trade-off between exact and approximate symmetry enforcement, showing that approximate equivariance is often sufficient to obtain the statistical and optimization benefits of full symmetrization while being dramatically cheaper. Using tools from harmonic analysis and representation theory, we show that small random subsets of transformations can match the generalization and convergence guarantees of full symmetrization up to controllable error. Our results identify a sharp separation between exact and approximate symmetry: enforcing approximate symmetry is exponentially cheaper than enforcing exact symmetry, formally justifying why approximate symmetry may be preferable in practice.