Remove the Ambiguity: Few-shot Multimodal Anomaly Detection Using Crossmodal Feature Replacers

A key challenge in reconstruction-based multimodal anomaly detection is the one-to-many crossmodal mapping problem: a single 3D feature may correspond to multiple plausible RGB appearances, causing deterministic crossmodal regression to collapse valid targets into over-smoothed reconstructions and thereby weaken anomaly discrimination. In this paper, we propose Crossmodal Feature Replacer (CFR), a self-supervised framework that addresses this failure mode through selective inference-time feature replacement. CFR first learns bidirectional cyclic mappings for coarse crossmodal reconstruction, then identifies unreliable reconstructed features and selectively replaces them with high-confidence normal features to correct ambiguity-induced reconstruction failures. Extensive experiments on MVTec 3D-AD and Eyecandies under few-shot settings show that CFR consistently outperforms prior methods. In the challenging 1-shot setting, CFR achieves AUPRO scores of 92.3 and 82.7 at 30\% FPR, together with image-level AUROC scores of 74.0 and 75.9, on MVTec 3D-AD and Eyecandies, respectively. Code is available at https://github.com/Yuan-Honoka-Guo/CFR.

Industrial inspection systems often need to find defects using both camera images and 3D shape information, but only a few normal examples may be available. A major difficulty is that the same 3D shape can have several reasonable visual appearances, such as different textures, colors, or lighting. When a model tries to directly predict the image features from the 3D features, it may average these possibilities together and produce overly smooth reconstructions, making defects harder to identify. We propose Crossmodal Feature Replacer (CFR), a method that first learns the relationship between image and 3D features, then detects unreliable reconstructed features during testing and replaces them with more reliable normal features. This helps correct reconstruction errors caused by ambiguous image–shape relationships. Experiments on two industrial anomaly detection benchmarks show that CFR improves few-shot defect detection, making it useful when collecting many normal training samples is costly or impractical.