Visual Para-Thinker: Divide-and-Conquer Reasoning for Visual Comprehension

Existing LLM test-time scaling laws emphasize the emergence of self-reflective behaviors through extended reasoning length. Nevertheless, this vertical scaling strategy often encounters plateaus in exploration as the model becomes locked into specific thinking pattern. By shifting from depth to parallelism, parallel thinking mitigates the narrowing of exploration. However, the extension of this paradigm to visual domain remains an open research question. In this paper, we first examine the role of visual partitioning in parallelized reasoning and subsequently propose two distinct strategies. Based on the above, we introduce Visual Para-Thinker, representing the inaugural parallel reasoning framework for MLLMs. To maintain path independence and promote diversity in reasoning, our approach integrates Pa-Attention alongside LPRoPE. Leveraging the vLLM framework, we have developed a native multimodal implementation that facilitates high-efficiency parallel processing. Empirical results on benchmark datasets such as V*, CountBench, RefCOCO, and HallusionBench confirm that Visual Para-Thinker successfully extends the benefits of parallel reasoning to the visual domain.

Current AI models often struggle with complex visual tasks because they "think" in a single, narrow line, which frequently leads them to get stuck in repetitive or incorrect reasoning patterns. We developed "Visual Para-Thinker," a new approach that allows AI to break down visual information and explore multiple lines of reasoning at the same time, rather than just one. By enabling this "parallel thinking," our method helps AI models solve tricky visual puzzles more accurately and reliably, paving the way for smarter and more efficient multimodal AI.