AlignVid: Taming Visual Dominance via Training-Free Attention Modulation in Text-guided Image-to-Video Generation

Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (e.g., object addition, deletion, or modification). Empirically, our analysis reveals that this stems from visual dominance, where the reference image causes severe attention dispersion, inhibiting the model's ability to incorporate new semantic information. To address this, we propose AlignVid, a training-free intervention that re-calibrates the model's internal attention distribution. Drawing on an energy-based perspective of attention, AlignVid employs Attention Scaling Modulation (ASM) to reduce attention entropy and concentrate focus on semantic tokens, alongside Guidance Scheduling (GS) to maintain generation stability. To rigorously assess this capability, we present OmitI2V, a comprehensive benchmark for evaluating prompt adherence across object addition, deletion, and modification. Extensive experiments demonstrate that AlignVid effectively enhances semantic fidelity with negligible computational overhead.

Problem. Today's I2V models can turn a still image and a simple instruction—such as “make the cat wear sunglasses” or “grow a sunflower from the soil”—into a short video. Yet even strong image-to-video models often animate what is already in the picture while missing the requested change in the text. Solution. We identify this failure as visual dominance: inside the model, the input image overwhelms the text signal, causing the generated video to stay too close to the original picture. AlignVid fixes this with a lightweight, training-free intervention applied during generation. Instead of blurring or altering the input image, it directly rebalances the model’s internal attention, reducing over-reliance on visual tokens and strengthening text guidance. The method requires no retraining, adds no new modules, and introduces almost no extra computation. Impact. Across three state-of-the-art video generation systems, AlignVid consistently improves whether the requested object addition, removal, or modification appears in the final video while preserving visual quality. We also introduce OmitI2V, a human-curated benchmark of 367 challenging cases designed to test whether text-guided image-to-video models truly follow edit instructions rather than merely animate the input image.