Don't Overthink with Pixels: Efficient Reasoning for Segmentation

Existing reasoning segmentation approaches typically fine-tune multimodal large language models (MLLMs) using image-text pairs and corresponding mask labels. While recent efforts leverage reinforcement fine-tuning to further enhance reasoning ability, they often suffer from overthinking and produce uniformly verbose reasoning chains irrespective of task complexity. To address this problem, we propose PixelThink, a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty to regulate reasoning generation within a reinforcement learning paradigm. The model learns to compress reasoning length in accordance with scene complexity and predictive confidence. To support comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark with annotated reasoning references and difficulty scores, along with a suite of metrics designed to assess segmentation accuracy, reasoning quality, and efficiency jointly. Experimental results demonstrate that the proposed approach not only improves segmentation performance but also significantly reduces inference latency by 30.4%, cutting token usage by 48.2%.

Artificial intelligence models designed to locate and highlight specific objects in images based on text instructions have become highly advanced. However, the latest generation of these models suffers from a tendency to "overthink." They frequently generate unnecessarily lengthy and complex explanations even for the simplest tasks, resulting in a significant waste of time and computational resources. To address this inefficiency, we introduce PixelThink. This approach trains the AI to adapt its reasoning process based on the difficulty of the task: it provides quick, direct answers for simple queries, and reserves detailed, step-by-step explanations only for truly complex images or instructions. To evaluate this effectively, we developed a comprehensive scoring system that assesses the AI not only on its accuracy in finding the correct object but also on its operational efficiency. Our results demonstrate that PixelThink enhances the model's accuracy while accelerating the entire process by over 30% and reducing computational token usage by nearly half.