Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models

Large diffusion vision–language models (LDVLMs) have recently emerged as a promising alternative to autoregressive models, enabling parallel decoding for efficient inference and leveraging bidirectional attention for global context. Despite these advances, their behavior under long-form generation remains underexplored. In this work, we show that existing LDVLMs suffer from repetitive generation and degraded visual grounding, and identify two underlying causes. First, repetitive generation originates from a mask token prior: since generation tokens are initialized as mask tokens, their hidden representations progressively drift toward a shared prior direction over generation steps. Second, a fundamental misalignment between the positional attention bias and the iterative unmasking process suppresses attention toward informative visual tokens, degrading visual grounding. Based on these insights, we propose a training-free approach, introducing Mask Prior Suppression and Monotonic RoPE Scaling to mitigate mask prior drift and positional attention collapse during decoding. Experiments on general multimodal benchmarks and visual grounding tasks demonstrate improvements over baseline LDVLMs, with robust gains on long-form description benchmarks. Our results show that these failures can be effectively addressed with a lightweight, plug-and-play strategy that requires no additional training and generalizes across diverse LDVLM architectures.

Most AI systems that describe images generate text one word after another. A newer approach called diffusion takes a different route: it starts with many blanks and fills them in parallel, which is much faster. But when asked for longer descriptions, these diffusion models tend to repeat themselves and lose track of what's actually in the picture. We traced this problem to two causes. First, every blank begins as the same generic placeholder, so as the model fills them in, its answers drift toward looking alike. Second, the model's sense of "where to look" was designed for one-word-at-a-time writing, so during parallel generation, it accidentally stops paying attention to the image. We developed two simple fixes, each directly targeting one of these causes. They require no retraining and work across different model architectures. On standard benchmarks, especially long-form description tasks, our method produces noticeably better and less repetitive results.