Efficient Multi-modal Dataset Distillation via Analytic Parameter Matching

Multi-modal dataset distillation (MDD) seeks to compress large-scale multi-modal datasets into a compact set of synthetic pairs. Existing methods employ a dual-trajectory matching framework to align the teacher and student models within each modality. While effective, this paradigm incurs non-negligible memory and computational overhead due to the checkpoint storage and bi-level optimization over synthetic data. To address these limitations, we propose analytic parameter matching (APM), which theoretically derives the analytic parameters of modal projectors to replace the inner-loop optimization, and then aligns the analytic projector parameters of teacher and student models. APM offers two key advantages: (1) it replaces checkpoint-intensive storage with only two cached matrices, significantly reducing memory consumption; and (2) it computes analytic parameters in a single forward pass, thereby avoiding costly bi-level optimization. Empirically, APM achieves up to 65$\times$ storage reduction and 9.6$\times$ faster distillation, while scaling to 1,000 synthetic pairs. Extensive experiments on image-text and audio-text benchmarks demonstrate the effectiveness of APM in cross-modal retrieval tasks, e.g., 12.8 IR@1 and 17.8 TR@1 in Flickr30k with 100 synthetic pairs. Moreover, APM exhibits notable generalization performance in cross-architecture evaluation and zero-shot classification tasks.

Modern AI systems often learn from paired data, such as images with captions or sounds with descriptions. Training on large collections of such data can be costly in storage, time, and computing power. This paper studies how to replace a large dataset with a much smaller set of artificial examples that still preserves its most useful information. Existing methods are expensive because they need to store many training records and repeatedly simulate the training process. We propose a simpler approach that directly estimates what information should be kept, making the compression process faster and more storage-efficient. Experiments on image-text and audio-text tasks show that our method reduces cost while maintaining strong performance, making multi-modal AI training more practical when resources are limited.