Navigating the Flatlands: Dual Adaptive Sharpness-Aware Minimization for Domain Generalization

Finding flat minima in the loss landscape is a key strategy for Domain Generalization (DG). However, its effectiveness is often limited by two crucial challenges. 1) Domain Shift: Existing methods like Sharpness-Aware Minimization (SAM) apply a uniform optimization strategy across all domains, overlooking the differences of the learning difficulties among multiple domains and thus performing poorly on challenging domains. 2) Anisotropic Sharpness: By perturbing parameters along a single gradient direction, SAM and its variants ignore multi-directional flatness, making the model converge to minima that remain sharp in other directions. The combined challenges make it more difficult for the model to find truly robust solutions in multi-domain scenarios. To overcome these limitations, we propose the Dual Adaptive Sharpness-Aware Minimization (DA-SAM), which comprises two key modules: Dynamic Adaptive Scaling (DAS) module and Adaptive Multi-Directional Flattening (AMDF) module. First, to tackle the domain shift problem, the DAS module computes the real-time loss on each domain to adaptively generate domain-specific scaling factors that guide the generation of perturbation directions. Second, the AMDF module calculates local flatness by generating multiple directions to simulate perturbations in the parameter space. Based on the learned local flatness metric, it dynamically adjusts the perturbation step size to guide the model parameters to be away from anisotropic sharp regions. Crucially, DAS provides domain-level guidance that makes AMDF’s multi-directional geometric exploration more targeted and effective. Extensive experiments on five DG benchmarks demonstrate the effectiveness of our DA-SAM algorithm.