Timezone: »
The Dual Principal Component Pursuit (DPCP) method has been proposed to robustly recover a subspace of high-relative dimension from corrupted data. Existing analyses and algorithms of DPCP, however, mainly focus on finding a normal to a single hyperplane that contains the inliers. Although these algorithms can be extended to a subspace of higher co-dimension through a recursive approach that sequentially finds a new basis element of the space orthogonal to the subspace, this procedure is computationally expensive and lacks convergence guarantees. In this paper, we consider a DPCP approach for simultaneously computing the entire basis of the orthogonal complement subspace (we call this a holistic approach) by solving a non-convex non-smooth optimization problem over the Grassmannian. We provide geometric and statistical analyses for the global optimality and prove that it can tolerate as many outliers as the square of the number of inliers, under both noiseless and noisy settings. We then present a Riemannian regularity condition for the problem, which is then used to prove that a Riemannian subgradient method converges linearly to a neighborhood of the orthogonal subspace with error proportional to the noise level.
Author Information
Tianyu Ding (Johns Hopkins University)
Zhihui Zhu (Johns Hopkins University)
Rene Vidal (Johns Hopkins University, USA)
Daniel Robinson (Lehigh University)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Dual Principal Component Pursuit for Robust Subspace Learning: Theory and Algorithms for a Holistic Approach »
Fri. Jul 23rd 02:45 -- 02:50 AM Room
More from the Same Authors
-
2023 Workshop: HiLD: High-dimensional Learning Dynamics Workshop »
Courtney Paquette · Zhenyu Liao · Mihai Nica · Elliot Paquette · Andrew Saxe · Rene Vidal -
2023 Poster: On the Convergence of Gradient Flow on Multi-layer Linear Models »
Hancheng Min · Rene Vidal · Enrique Mallada -
2023 Poster: Learning Globally Smooth Functions on Manifolds »
Juan Cervino · Luiz Chamon · Benjamin Haeffele · Rene Vidal · Alejandro Ribeiro -
2023 Poster: The Ideal Continual Learner: An Agent That Never Forgets »
Liangzu Peng · Paris Giampouras · Rene Vidal -
2022 Poster: Understanding Doubly Stochastic Clustering »
Tianjiao Ding · Derek Lim · Rene Vidal · Benjamin Haeffele -
2022 Spotlight: Understanding Doubly Stochastic Clustering »
Tianjiao Ding · Derek Lim · Rene Vidal · Benjamin Haeffele -
2022 Poster: On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features »
Jinxin Zhou · Xiao Li · Tianyu Ding · Chong You · Qing Qu · Zhihui Zhu -
2022 Poster: Robust Training under Label Noise by Over-parameterization »
Sheng Liu · Zhihui Zhu · Qing Qu · Chong You -
2022 Spotlight: Robust Training under Label Noise by Over-parameterization »
Sheng Liu · Zhihui Zhu · Qing Qu · Chong You -
2022 Spotlight: On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features »
Jinxin Zhou · Xiao Li · Tianyu Ding · Chong You · Qing Qu · Zhihui Zhu -
2022 Poster: Reverse Engineering $\ell_p$ attacks: A block-sparse optimization approach with recovery guarantees »
Darshan Thaker · Paris Giampouras · Rene Vidal -
2022 Spotlight: Reverse Engineering $\ell_p$ attacks: A block-sparse optimization approach with recovery guarantees »
Darshan Thaker · Paris Giampouras · Rene Vidal -
2021 Poster: Understanding the Dynamics of Gradient Flow in Overparameterized Linear models »
Salma Tarmoun · Guilherme Franca · Benjamin Haeffele · Rene Vidal -
2021 Poster: On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks »
Hancheng Min · Salma Tarmoun · Rene Vidal · Enrique Mallada -
2021 Spotlight: On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks »
Hancheng Min · Salma Tarmoun · Rene Vidal · Enrique Mallada -
2021 Spotlight: Understanding the Dynamics of Gradient Flow in Overparameterized Linear models »
Salma Tarmoun · Guilherme Franca · Benjamin Haeffele · Rene Vidal -
2021 Poster: A Nullspace Property for Subspace-Preserving Recovery »
Mustafa D Kaba · Chong You · Daniel Robinson · Enrique Mallada · Rene Vidal -
2021 Spotlight: A Nullspace Property for Subspace-Preserving Recovery »
Mustafa D Kaba · Chong You · Daniel Robinson · Enrique Mallada · Rene Vidal -
2020 : Spotlight talk 1 - A Second-Order Optimization Algorithm for Solving Problems Involving Group Sparse Regularization »
Daniel Robinson -
2019 Poster: Alternating Minimizations Converge to Second-Order Optimal Solutions »
Qiuwei Li · Zhihui Zhu · Gongguo Tang -
2019 Poster: Noisy Dual Principal Component Pursuit »
Tianyu Ding · Zhihui Zhu · Tianjiao Ding · Yunchen Yang · Daniel Robinson · Manolis Tsakiris · Rene Vidal -
2019 Oral: Alternating Minimizations Converge to Second-Order Optimal Solutions »
Qiuwei Li · Zhihui Zhu · Gongguo Tang -
2019 Oral: Noisy Dual Principal Component Pursuit »
Tianyu Ding · Zhihui Zhu · Tianjiao Ding · Yunchen Yang · Daniel Robinson · Manolis Tsakiris · Rene Vidal -
2018 Poster: ADMM and Accelerated ADMM as Continuous Dynamical Systems »
Guilherme Franca · Daniel Robinson · Rene Vidal -
2018 Oral: ADMM and Accelerated ADMM as Continuous Dynamical Systems »
Guilherme Franca · Daniel Robinson · Rene Vidal