Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design

Neural architecture design lacks first principles: innovations are discovered empirically and justified post-hoc, with no systematic way to diagnose *why* an architecture fails or derive *what* repair will succeed. We introduce the *Axiomatic Atlas*, encoding requirements as composable axioms over graph connectivity, operator contracts, numerical stability, and information preservation. Given an operator library and wiring conventions, the Atlas constructs certificates lower-bounding output variation via min-cut analysis and diagnoses failures by locating axiom violations. Crucially, the framework is prescriptive: each violation implies a targeted repair, reducing architecture design to constraint satisfaction. We prove variation bounds under exact and finite-precision arithmetic, enabling modular verification across transformers, MoEs, SSMs, and GNNs. Four Atlas-derived interventions validate the approach: +46 percentage points on GNN bottlenecks, $3\times$ robustness to MoE quantization, 83\% gap closure with adaptive expert budgets, and 0\%$\to$100\% retrieval via orthogonal keys---each against matched negative controls.

Neural network architecture design today is mostly trial and error: when something fails, engineers guess at a fix and hope it works. We introduce the Axiomatic Atlas, a framework that replaces this guesswork with a systematic checklist of requirements a well-functioning architecture should satisfy. These requirements cover whether information flows without bottlenecks, whether components stay responsive, whether signals survive through many processing layers, and whether information is preserved when parallel pathways merge. Given any architecture, the Atlas runs an audit that produces a health report and a diagnosis pinpointing what is wrong and where. Each type of failure implies a specific repair, turning open-ended search into targeted problem-solving. We validated this on four architecture families: the Atlas correctly diagnosed each failure and guided fixes that substantially outperformed generic alternatives — including a 46-percentage-point accuracy gain from strategically placed information shortcuts, and a jump from 0% to 100% retrieval accuracy by correcting a geometric interference problem. Control experiments confirmed these gains came from fixing the specific diagnosed problem. The Atlas does not guarantee task performance, but it provides a principled way to find and fix architectural problems, potentially saving significant time and computation.