Position: Weight Space Should Be a First-Class Generative AI Modality

Neural network checkpoints have quietly become a large-scale data resource: millions of trained weight vectors now exist, each encoding task-, domain-, and architecture-specific knowledge. This position paper argues that model checkpoints should be treated as a first-class data modality, and that generative modeling in weight space should be standardized as a core machine learning primitive. Recent advances demonstrate that neural weights can be synthesized on demand, often matching fine-tuning performance while reducing adaptation cost by orders of magnitude. We contend that these results reflect an underlying structural fact: high-performing models occupy low-dimensional, highly structured regions of weight space shaped by symmetry, flatness, modularity, and shared subspaces. Building on this view, we organize existing methods into a standardized five-stage pipeline for weight-space generation and survey applications where the approach is already practical, such as parameter-efficient adaptation, mid-scale model synthesis, and on-device learning. We then confront alternative views, clarify current limits, and issue a concrete call to action. Our goal is to shift the community’s default mindset from optimizing models per task to sampling models from learned weight distributions, accelerating toward an era in which AI systems routinely generate other AI systems.