BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

Today’s AI models are great at describing general scenes but struggle to find tiny objects in massive, high-resolution images—like spotting a single bird in a vast forest. Current methods either scan the entire image too slowly or fail to correct themselves if their first guess is wrong. We developed BVS (Bayesian Visual Search), a framework that acts like a "smart, self-correcting magnifying glass." BVS first uses the AI’s internal "gut feeling" to identify likely areas of interest. It then uses a mathematical optimization strategy to zoom in on these spots while dynamically searching for missing information and correcting initial errors. We also introduced a "scale-aware" tool to help the AI maintain focus even as the perspective changes during zooming. Our experiments show that BVS allows AI to find objects occupying as little as 0.2% of an image much more accurately and efficiently. This research paves the way for AI that can reliably understand complex, high-definition environments, from medical imaging to satellite surveillance.