The Implicit Bias of Depth: From Neural Collapse to Softmax Codes

Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architectures, attributing this to an explicit low-rank bias from L2 regularization. We study the deep unconstrained feature model (UFM)—equivalent to a deep linear network with orthogonal inputs—trained without regularization, to isolate how gradient descent and depth alone shape NC. We show that depth induces an implicit low-rank bias: low-rank matrices propagate norm more efficiently through successive multiplications, promoting low-rank alternatives to NC. These alternatives, we argue, correspond to softmax codes: max-margin solutions previously found in width-bottlenecked networks. Analyzing training dynamics under spectral initialization, we identify an early-time repulsion among singular values that drives low-rank emergence, and characterize how depth shrinks NC's basin of attraction. Finally, we show that some effects act in the opposite direction: for randomly initialized networks, increasing width biases training toward higher-rank solutions. Our results provide the first asymptotic and dynamic characterization of implicit bias in deep UFMs trained with unregularized multiclass cross-entropy.

Modern neural networks often learn structured internal representations when trained for classification: examples from the same class cluster together, and different classes arrange themselves in a highly symmetric pattern known as neural collapse. This pattern is widely observed, but it is still unclear which parts of training and architecture cause it to appear. We study a mathematically tractable model of deep classifiers to isolate the effect of network depth itself. We find that depth can push training away from neural collapse and toward simpler lower-dimensional alternatives. These alternatives are known as “softmax codes,” geometric arrangements previously associated with networks that are explicitly forced to use narrow hidden representations. We also show that training dynamics and initialization matter: early in training, depth can encourage low-rank structure, while large width networks are biased back toward higher-rank, neural-collapse-like solutions. Experiments on simplified models and standard image datasets support these predictions. Overall, our work helps explain when neural collapse should be expected, when depth may prevent it, and how architecture and optimization shape the geometry learned by classifiers.