Set-Coupled Guidance: Set-Level Coordination in Diffusion-Based Dataset Distillation

Diffusion models serve as generative priors for dataset distillation, yet existing pipelines rely on per-sample update rules that evolve each synthetic image independently, limiting their ability to optimize collective set-level objectives. We propose Set-Coupled Guidance (SCG), a plug-and-play auxiliary controller that shifts from per-image to group (IPC-at-once) sampling by injecting set-symmetric feedback at each diffusion step. SCG combines spectral set-point regulation, which aligns set-level statistics to real data via empirical characteristic function matching, with cooperative kernel coupling that stabilizes joint trajectories under noisy feedback. All computations operate on lightweight descriptors extracted from predicted clean latents, adding low overhead to the base method. We provide theoretical analysis including Lyapunov descent and input-to-state stability for distributional tracking. Experiments on ImageNette, ImageWoof, ImageNet-100 and ImageNet-1K show consistent accuracy gains across multiple diffusion-based baselines; code is available at \url{https://github.com/tade1s/SCG}.

Training image recognition systems often requires very large datasets, which can be expensive to store, process, and reuse. Dataset distillation tries to solve this by creating a much smaller synthetic dataset that still teaches a model much of what it would learn from the original data. Recent methods use diffusion models to generate these synthetic images, but they usually guide each image separately. This can lead to repeated or poorly coordinated examples, even though the final result is judged as a set of images. This paper introduces Set-Coupled Guidance, a lightweight add-on for existing diffusion-based dataset distillation pipelines. Rather than replacing the underlying generator or distillation method, it watches the synthetic images from the same class as a group while they are being generated and gently adjusts their updates so the group covers the class more evenly. The add-on does not require retraining the diffusion model and does not add extra data to the final distilled set. In experiments, adding it to several existing methods improves accuracy across image benchmarks, suggesting a practical way to make small synthetic training sets less redundant and more useful.