Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization's missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL's objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95\%$), and match CE's converged NC on 4/5 metrics in under $7.5\%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5\%$ mean relative improvement in transfer learning, up to $+8.7\%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached *by design*.

When we want to classify different objects into different categories, say a table from a helmet, we actually want examples of each category grouped tightly and different categories kept well apart. It is proven that there is one ideal geometric arrangement that does this perfectly, known as Neural Collapse: each category gets an "anchor point," all examples of that category cluster around it, and these anchors sit maximally far from each other, like equally spaced cities on a globe. Despite this elegant theory, the two most popular methods for training such classifiers almost never reach this ideal in practice. We show that these two methods, which look very different on the surface, are actually doing the same thing: both are learning an anchor point for each category, and each fails in its own specific way. We fix both by constraining the geometry to lie on a sphere, where the ideal arrangement is the only option, and we design new training objectives that provably reach it. Models trained this way are more accurate, transfer better to new tasks, handle imbalanced and corrupted data more gracefully, and skip an hours-long step that traditional methods normally require.