REVIS: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models

Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visual features and pretrained textual representations often become intertwined in the deeper network layers. To address this, we propose REVIS, a training-free framework designed to explicitly re-activate this suppressed visual information. Rooted in latent space geometry, REVIS extracts the pure visual information vector via orthogonal projection and employs a calibrated strategy to perform sparse intervention only at the precise depth where suppression occurs. This surgical approach effectively restores visual information with minimal computational cost. Empirical evaluations on standard benchmarks demonstrate that REVIS reduces object hallucination rates by approximately 19% compared to state-of-the-art baselines, while preserving general reasoning capabilities.

AI systems that understand both images and language, often called vision-language models, can answer questions about pictures, but they sometimes describe objects that are not actually present. This problem can make their answers misleading and reduce trust in applications that rely on visual understanding. We developed REVIS to help these models rely more on what is visible in the image. Instead of retraining the model or adding another model, REVIS makes a small targeted adjustment inside the model when it appears to rely too much on learned language patterns. Our experiments show that REVIS reduces these incorrect object descriptions while keeping the model’s general image understanding ability mostly unchanged. This makes vision-language AI systems more reliable for real-world use.