Noise-Robust Density Estimation for Tabular Data Anomaly Detection

Density-based anomaly detection methods often provide accurate and interpretable predictions but their performance can be severely degraded by the inherent noise of data, such as changes arising from environmental conditions during data collection or background noise. To deal with such noise, we present noise-robust density estimation (NRDE) for tabular data anomaly detection. We aim to estimate the density of pure data with the influence of noises isolated, which is a non-trivial task since the data-generating process is completely unknown. Specifically, NRDE learns a Jacobian-regularized normalizing flow to estimate the sources of data and categorizes sources into two groups, where one group generates pure data and the other generates noise. After generating pure data, we can use the density of such pure data to detect anomalies caused by the sources of pure data solely. Therefore, NRDE is robust to inherent noise. We provide theoretical results to support the effectiveness of NRDE and compare NRDE with $17$ baselines on $47$ benchmark datasets under different settings, including vanilla anomaly detection, anomaly detection with anomaly contamination, anomaly detection on noisy data, and transductive outlier detection.

Many important decisions rely on finding unusual or anomalous records in data tables, such as possible fraud, machine failures, or health-related warning signs. However, real-world data often contains harmless noise: for example, changes in the environment, measuring devices, sensors, or background conditions. Existing methods may mistake these harmless changes for real problems, causing many false alarms. This paper studies how to detect truly unusual records while ignoring such harmless noise. Our method tries to separate the meaningful part of each record from the noisy part, and then decides whether the meaningful part looks abnormal. In this way, a record is not treated as suspicious simply because it was affected by background variation. We test the method on 47 datasets and compare it with 17 existing approaches under several practical settings, including noisy data and data that already contains some unusual records. The results show that the proposed method is more reliable in the presence of noise and can better distinguish real anomalies.