Self-Prophetic Decoding to Unlock Visual Search in LVLMs

Large Vision-Language Models (LVLMs) are rapidly evolving toward true multimodal reasoning, with visual search representing a concrete instantiation of the thinking-with-images paradigm. However, LVLM visual search faces two key challenges: incompatibility among intrinsic capabilities after post-training, and interference in long multi-step reasoning contexts. To address these, we identify two novel insights. First, self-regulation between pre- and post-training LVLMs leverages the intrinsic single-step capabilities of the pre-training model to mitigate capability deterioration and long-context interference. Second, probability-based prophetic sampling, replacing naive prompting, provides a probabilistic interface where the pre-training model acts as a prophet and the post-training model selectively accepts prophetic tokens under its output distribution, preserving coherent multi-step reasoning. Building on these insights, we introduce SeProD, a self-prophetic decoding framework that leverages intrinsic single-step capabilities to enable coherent multi-step reasoning in a training-free, plug-and-play manner. Experiments show that SeProD consistently improves multiple visual-search LVLMs across all 12 splits of 4 visual search benchmarks, as well as across general VQA benchmarks, without added computational overhead, thanks to its parallel prophetic acceptance mechanism.

Large Vision-Language Models (LVLMs) are becoming increasingly capable of visual reasoning, especially for visual search tasks that require multi-step interactions with images. However, we find that post-training can weaken the model’s original intrinsic single-step capabilities and make reasoning unstable in long contexts. We introduce SeProD, a training-free and plug-and-play framework that allows a post-training LVLM to collaborate with its original pre-training model during inference. Instead of relying on naive prompting, SeProD uses probability-based prophetic sampling: the pre-training model proposes candidate tokens, while the post-training model selectively accepts them according to its own output probabilities. This helps recover intrinsic capabilities while maintaining coherent multi-step reasoning. SeProD consistently improves multiple LVLMs across all 12 splits of 4 visual-search benchmarks, as well as on general visual question answering benchmarks, without adding computational overhead. Our results suggest that coordinating pre- and post-training models with an effective interface during decoding can improve the robustness and reliability of multimodal reasoning systems.