Adversarial Vulnerability from Interference Between Features in Superposition

Why do adversarial examples exist, and why do they transfer between models? Existing explanations appeal to high-dimensional geometry, non-robust patterns in the input, and decision boundary structure, but none provides a representation-level mechanism that explains why specific perturbations succeed and why attacks transfer between models. In this paper, we show that adversarial vulnerability can stem from efficient information encoding in neural networks. Specifically, vulnerability can arise from superposition - the phenomenon where networks represent more concepts than they have dimensions, forcing non-orthogonal representation and thus interference. This interference causes perturbations targeting one representation to affect others, creating vulnerabilities determined by interference patterns. In synthetic settings with precisely controlled superposition, we establish that superposition suffices to create adversarial vulnerability. The resulting attacks are predictable: PGD-discovered perturbations align with theoretically optimal perturbations derived from the interference geometry. Models trained on similar data develop similar interference patterns, explaining attack transferability. We then show that successful attacks on image classifiers exhibit the structure predicted by our proposed mechanism. These findings reveal that adversarial vulnerability can be a byproduct of networks' representational compression, complementing existing explanations based on data properties or architectural factors.

AI models can be fooled by tiny, imperceptible changes to a picture - a few carefully chosen tweaks that make a system confidently call a cat a dog. These 'adversarial examples' are easy to create but hard to explain, and stranger still, an attack designed to fool one network often fools a different one. We wanted to understand why. Our explanation comes from how networks represent concepts internally. To be efficient, they cram in more concepts, so concepts are forced to overlap and share the same internal 'wiring'. We show that attackers can exploit this. Because concepts interfere with one another, a nudge aimed at one can quietly impact others. In simple, controlled models, this overlap alone creates these vulnerabilities - and we can predict, in advance, which attacks will succeed and which will carry over between models. We then find the same fingerprints in realistic image classifiers. This reframes a notorious weakness as the price networks pay for storing information efficiently - suggesting robustness and capacity are in tension, and that defences should consider how networks pack concepts together rather than just the attacks themselves.