Parameter Manifold Purification

Deep models are vulnerable to performance degradation caused by various factors, such as imbalanced samples, inaccurate labels, and backdoor attacks. However, existing optimization methods that address these issues are typically designed in a scenario- or architecture-specific manner, and each optimization often requires costly training. To this end, inspired by image denoising, we propose parameter purification as a new paradigm for model performance optimization. Parameter purification attributes performance degradation to the contamination of model parameters and aims to recover clean parameters from corrupted ones in a manner analogous to image denoising. To purify parameters with massive scale and complex structure, we further introduce a novel parameter manifold purification method. In this framework, high-dimensional and complex parameters are first viewed as manifolds embedded in a high-dimensional space, and are then partitioned into nested local parameter-cluster manifolds via a proposed parameter clustering strategy. Meanwhile, to remove parameter redundancy while preserving global parameter information, we propose an implicit manifold auto-encoder along with a parameter-cluster discrepancy loss to learn low-dimensional representations of parameter-cluster manifolds. Finally, an implicit conditional diffusion model is applied to denoise the low-dimensional parameter manifolds, progressively restoring clean parameters. Extensive experiments under three representative scenarios that cause model performance degradation demonstrate that parameter manifold purification can accurately and completely purify corrupted parameters of unseen models, analogous to denoising unseen images, and rapidly improve model performance.

Modern AI models can perform poorly when their training data is unbalanced, labels are wrong, or hidden attacks have changed their behavior. This paper proposes a new way to fix such models by directly repairing their internal parameters, much like removing noise from a damaged image. Instead of retraining the whole model for each problem, our method breaks the model parameters into smaller meaningful groups, learns what clean parameters should look like, and then restores corrupted ones through a denoising process. Experiments show that this approach can improve accuracy and reduce harmful effects across several common failure cases, suggesting a practical path toward making AI models more reliable and easier to repair.