Low-Rank and Sparsity Are All You Need: Exploring Robust Hierarchical Latent Subspaces for Transferable Adversarial Attack

Adversarial examples pose serious threats to deep neural networks, exposing fundamental vulnerabilities in model robustness. However, most existing adversarial attacks directly manipulate dense and redundant feature representations, often leading to overfitting on surrogate models and poor black-box transferability. Recent SVD-based attack attempts to exploit low-rank feature subspaces, yet its reliance on single-layer optimization and single-gradient pathway neglects structural redundancy in feature representations and hierarchical heterogeneity across layers. To address these limitations, we propose LRS-Attack, a low-rank and sparse decomposition attack that explicitly models robust hierarchical subspaces in latent feature spaces. Specifically, the low-rank component captures dominant semantic directions, while the sparse component captures localized and discriminative patterns. To efficiently extract low-rank structure while preserving subspace fidelity, we develop a warm-started alternating low-rank approximation algorithm. Moreover, we introduce a hierarchical mixture of robust experts that leverages depth-dependent feature characteristics and guides gradient optimization toward more transferable adversarial directions. Extensive experiments on ImageNet show that LRS-Attack consistently improves black-box transferability over state-of-the-art methods across diverse CNN/ViT architectures and defense settings. Code is available at https://github.com/AdvML-Group/LRS-Attack.

Deep neural networks can be fooled by carefully crafted adversarial examples, raising concerns about the reliability and security of modern AI systems. Existing transferable adversarial attacks often operate on highly redundant feature representations, which may cause the generated perturbations to overfit to specific surrogate models and become less effective against unseen models. In this work, we propose LRS-Attack, a novel adversarial attack framework that explores the internal structure of neural network feature representations through low-rank and sparse decomposition. Intuitively, the low-rank component captures globally shared semantic information, while the sparse component focuses on localized and discriminative details. By jointly leveraging these two complementary components across multiple network layers, our method generates adversarial perturbations that transfer more effectively across different architectures and defense settings. We further design a hierarchical mixture-of-experts strategy to adaptively exploit feature characteristics from shallow to deep layers, further improving the transferability of adversarial examples. Extensive experiments on ImageNet demonstrate that LRS-Attack consistently outperforms existing state-of-the-art transferable attacks across a wide range of CNN and ViT models, as well as defended models and defense methods. Our findings highlight the importance of structured feature representations in adversarial transferability and provide new insights into the robustness and security of deep learning systems.