Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models
Abstract
Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding. As AI-generated images become realistic, semantic-level inconsistencies alone are often insufficient for reliable detection. This motivates a critical question: whether MLLMs can achieve full-spectrum forensic signal perception, i.e., capturing low-level generator artifacts without sacrificing pre-trained semantic knowledge. We then conduct a layer-wise analysis of forensic signal perception in MLLMs and find that semantic information is mainly encoded in the early-to-middle layers, and directly fine-tuning MLLMs for artifact learning causes rapid semantic forgetting. Based on this insight, we propose Deep Visual Residual MLLM (Deep-VRM) to \textit{preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer}, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input, achieving robust and generalizable detection performance. Extensive experiments show that our method achieves state-of-the-art across all benchmarks.
Lay Summary
As AI-generated images become more realistic, it is no longer enough to look only for obvious mistakes. To decide whether an image is real or fake, a detector often needs to use two kinds of clues. The first kind is semantic clues, which are problems in the meaning or content of an image. These are mistakes that people can often understand by looking carefully. For example, a car may have a missing side mirror, a person may have an unnatural hand, or text on a sign may be distorted and unreadable. These clues are related to whether the scene makes sense. The second kind is tiny visual traces, which are much harder for people to see. These are small patterns left by the image generation process, such as unusual textures, slight pixel-level irregularities, or subtle digital artifacts in smooth regions like skin, sky, walls, or road surfaces. Even when an AI-generated image looks reasonable overall, these traces may still reveal that it was created by a generator. This paper studies how multimodal large language models can better detect both kinds of clues. We find that these models are good at understanding image content, but they often miss tiny visual traces. However, directly forcing the whole model to learn these traces can harm its original ability to understand the image. To solve this problem, we propose Deep-VRM. The key idea is to let the model first keep its normal image understanding, and then add the tiny visual trace information at a later stage. In this way, the model can combine high-level content clues with fine visual evidence. Experiments on many datasets show that Deep-VRM detects AI-generated images more reliably, including images shared online that may have been resized or compressed.