ProtoVAR: Efficient Dataset Distillation via Prototype-Guided Visual Autoregressive Modeling

Recent advances in generative distillation have shown strong potential in constructing high quality surrogate datasets within a fraction of the time required by optimization-based approaches. However, most existing generative solutions rely on diffusion models, which suffer from two limitations. (i) Indirect matching objectives. Their sequential denoising process makes it difficult to directly match representative prototypes. (ii) Target-agnostic generation. The generation process is often decoupled from the target task, causing the synthesized samples to drift from the desired distribution. Building on this insight, We propose ProtoVAR, a prototype-guided visual autoregressive framework. Instead of relying on latent space, ProtoVAR uses the coarse-to-fine next-scale prediction of Visual AutoRegressive (VAR) modeling to maintain semantic consistency during generation. By injecting multi-scale class prototypes, ProtoVAR enforces clear representativeness constraints while preserving diversity. A pool-based selector further distills the prototype-guided outputs into a compact, task-aligned surrogate dataset. Extensive experiments show that ProtoVAR achieves state-of-the-art performance with comparable or lower computational cost than diffusion-based distillation.

As massive amounts of data lead to heavy computational and storage demands, improving efficiency through data reduction has become a critical challenge. Dataset distillation addresses this challenge by compressing a large dataset into a much smaller version that can still train models effectively. However, current methods face a major trade-off. Traditional coreset methods select real samples that represent the data well, but they severely lack variation. Meanwhile, newer diffusion-based generative methods are computationally expensive and struggle to balance sample diversity with accurate class representation. To solve this, we introduce ProtoVAR, an efficient framework that moves away from traditional diffusion models. Instead, we leverage Visual AutoRegressive modeling, guided by a small set of representative prototypes from the original data. By using these prototypes to steer the image generation process from coarse layouts down to fine details, ProtoVAR ensures the synthetic images stay true to the target categories while capturing essential visual diversity. As a result, our approach achieves state-of-the-art accuracy across multiple benchmarks at a fraction of the computational cost of previous generative methods.