When Random Saliency Looks Trained: Architectural Center Bias in CNN Interpretability

Saliency maps are widely used to interpret image classification models and build trust in their predictions; however, their reliability remains a central concern, as randomized networks can produce saliency maps that closely resemble those of trained models. We identify a previously underappreciated architectural contributor to this phenomenon: a center-focused saliency bias induced by common convolutional design choices. Through controlled ablations, we show that architectural components such as zero padding and receptive field growth induce a center-focused saliency prior that is already present in randomly initialized CNNs and under randomized inputs. In contrast, this behavior is largely absent in non-convolutional architectures such as Vision Transformers (ViTs) and multilayer perceptrons (MLPs). To investigate the interaction between architectural priors and learning, we introduce a corner-shift benchmark and a Center-Shift Index that quantify how saliency redistributes under object relocation. We show that training can partially shift saliency toward object regions, while randomized models remain dominated by architectural center bias, providing one mechanism by which trained-random similarity can be inflated and clarifying how architectural priors can confound standard saliency evaluations.

Saliency maps are commonly used to visualize which parts of an image a neural network focuses on when making a prediction. However, previous studies found that even random networks can sometimes produce saliency maps that look surprisingly similar to those from trained models, raising concerns about whether these explanations are trustworthy. In this work, we show that part of this similarity comes from the architecture of convolutional neural networks (CNNs) themselves. Many CNNs naturally produce saliency maps that emphasize the center of the image, even before training and even on meaningless shuffled images. This happens because common design choices in CNNs create built-in spatial biases. We compare CNNs with other model families such as Vision Transformers and MLP-based models, and find that this center-focused behavior is much weaker outside convolutional architectures. We also introduce a simple benchmark that moves objects away from the image center to test whether saliency maps truly follow the object or mainly reflect architectural bias. Our results suggest that some existing methods for evaluating explanation quality can be unintentionally influenced by these built-in architectural effects. We hope this work encourages more careful evaluation of interpretability methods and helps researchers better separate learned behavior from architectural bias.