Mean-Shift PCA by Knockoff Mean

Removing noise is difficult, but adding noise is easy. In this work, we show how to eliminate mean-shift noisy components from PCA by deliberately introducing knockoff mean-shift perturbation. Standard PCA is highly sensitive to shifts in the sample mean: a small fraction of samples from a shifted distribution can cause large deviations in the leading principal components. In high-dimensional regimes, existing Robust PCA approaches cannot handle the mean-shift contamination structure inherent in the mixture model. Using tools from Random Matrix Theory, we prove that the mean-shift spikes are spectrally separable from the stable eigenvalues of the original covariance. Furthermore, the original eigenspace remains asymptotically invariant to the contamination, independent of the mixture weight. Exploiting this spectral stability, we propose a simple, two-stage PCA algorithm by adding knockoff mean that identifies and removes the mean-shift component using only standard PCA operations.

Many modern datasets have thousands of measurements per sample and are often pooled from different groups, instruments, or batches. A common tool called principal component analysis, or PCA, tries to find the main directions in which the data varies, but it can be fooled when even a small group of samples has a shifted average value. In that case, PCA may highlight the difference between groups or batches instead of the scientific signal researchers wanted to study. We study this failure mathematically and show that, in high-dimensional data, the false directions caused by shifted averages behave differently from the genuine directions of variation. Our method, Mean-Shift PCA, exploits this difference by deliberately adding a second artificial shift, running PCA again, and keeping only the directions that stay stable. This gives a simple two-step procedure that uses ordinary PCA calculations rather than heavier specialized methods. The result is a faster and more reliable way to use PCA on mixed or imperfect datasets, with potential benefits for fields where hidden group shifts can distort conclusions.