Unlocking Noise-Resistant Vision: Key Architectural Secrets for Robust Models Against Gaussian Noise

We investigate which architectural and input-pipeline choices make vision models more robust to additive Gaussian noise and convert these empirical insights into simple, actionable design rules. Specifically, we performed extensive evaluations on 1,174 pretrained vision models, empirically identifying four consistent factors for improved robustness against Gaussian noise: larger stem kernels, resolution-dependent downsampling under a pre-resize sensor-noise protocol, average pooling, and supervised vision transformers (ViTs) rather than CLIP ViTs, which yield up to 506 rank improvements and 21.6\%p accuracy gains. First, we prove that low-pass stem kernels attenuate noise with a gain that decreases quadratically with kernel size and that anti-aliased downsampling reduces noise energy roughly in proportion to the square of the downsampling factor. Second, we demonstrate that average pooling is unbiased and suppresses noise in proportion to the pooling window area, whereas max pooling incurs a positive bias that grows slowly with window size and yields a relatively higher mean-squared error and greater worst-case sensitivity. Third, we reveal and explain the vulnerability of CLIP ViTs via a pixel-space Lipschitz bound: The smaller normalization standard deviations used in CLIP preprocessing amplify worst-case sensitivity by up to 1.91 times relative to the Inception-style preprocessing common in supervised ViTs. Our results collectively disentangle robustness into interpretable modules, provide a theory that explains the observed trends, and build practical, plug-and-play guidelines for designing vision models more robust against Gaussian noise.

Computer vision systems can make mistakes when images contain random noise, such as the graininess caused by cameras or low-light conditions. This paper studies which design choices make image recognition models more reliable under this kind of noise. We tested 1,174 publicly available models and ran controlled experiments to isolate important factors. We found that models tend to handle noisy images better when they smooth information early, reduce noise during resizing, combine nearby information by averaging, and avoid input settings that unnecessarily amplify noise. These findings provide simple practical guidelines for choosing or designing vision models that are more dependable when real-world images are not perfectly clean.