Mechanistic Anomaly Detection via Functional Attribution

We can often verify the correctness of neural network outputs using ground truth labels, but we cannot reliably determine whether the output was produced by normal or anomalous internal mechanisms. Mechanistic anomaly detection (MAD) aims to flag these cases, but existing methods either depend on latent space analysis, which is vulnerable to obfuscation, or are specific to particular architectures and modalities. We reframe MAD as a functional attribution problem: asking to what extent samples from a trusted set can explain the model's output, where attribution failure signals anomalous behavior. We operationalize this using influence functions, measuring functional coupling between test samples and a small reference set via parameter-space sampling. We evaluate across multiple anomaly types and modalities. For backdoors in vision models, our method achieves state-of-the-art detection on BackdoorBench, with an average Defense Effectiveness Rating (DER) of 0.93 across seven attacks and four datasets (next best 0.83). For LLMs, we similarly achieve a significant improvement over baselines for several backdoor types, including on explicitly obfuscated models. Beyond backdoors, preliminary evidence shows our method can detect adversarial and out-of-distribution samples, and distinguishes multiple anomalous mechanisms within a single model. Our results establish functional attribution as an effective, modality-agnostic tool for detecting anomalous behavior in deployed models.

When a neural network produces an output, we can check whether it's correct, but not whether it got there through normal reasoning or some hidden, anomalous process. A backdoored image classifier might label a photo "airplane" because it sees wings and sky, or because an attacker planted a small visual trigger that overrides everything else. In each case, the output looks identical. Existing defences mostly inspect the model's internal activations, but attackers can learn to disguise these. We take a different angle: we ask whether the model is processing a test input the same way it processes a small set of inputs we already trust. We slightly jiggle the model's parameters and track how it's loss on each input moves. Inputs handled by the same internal mechanism move in sync, whereas inputs handled by different mechanisms drift apart. Test inputs that fail to track any trusted input get flagged. The method achieves state-of-the-art backdoor detection on standard image benchmarks and near-perfect detection on language models, including ones explicitly trained to evade activation-based defences. It also catches out-of-distribution inputs, adversarial examples, and can distinguish multiple hidden behaviours inside one model. As neural networks are deployed in higher-stakes settings, this offers a way to flag "this output came from somewhere unusual" without making assumptions attackers can game.