MotiMotion: Motion-Controlled Video Generation with Visual Reasoning

Current motion-controlled image-to-video generation models rigidly follow user-provided trajectories that are often sparse, imprecise, and causally incomplete. Such reliance often yields unnatural or implausible outcomes, especially by missing secondary causal consequences. To address this, we introduce MotiMotion, a novel framework that reformulates motion control as a reasoning-then-generation problem. To encourage causally grounded and commonsense-consistent interactions, we leverage a training-free vision-language reasoner to refine image-space coordinates of primary trajectories and to hallucinate plausible secondary motions. To further improve motion naturalness, we propose a confidence-aware control scheme that modulates guidance strength, enabling the model to closely follow high-confidence plans while correcting artifacts under low-confidence inputs with its internal generative priors. To support systematic evaluation, we curate a new image-to-video benchmark, MotiBench, consisting of interaction-centric scenes where new events are triggered by motion. Both VLM-based evaluation and a human study on MotiBench demonstrate that MotiMotion produces videos with more plausible object behaviors and interaction, and is preferred over existing approaches.

Many AI video generation systems let users control motion by drawing simple trajectories, such as arrows showing how an object should move. However, current systems often follow these instructions too literally. Because user inputs are usually sparse or incomplete, generated videos can look unnatural or fail to capture realistic consequences, such as dominoes falling after a hand moves away or gears rotating together. We introduce MotiMotion, a video generation framework that combines motion control with visual reasoning. Instead of treating user trajectories as exact commands, our system interprets them as high-level intentions. Using a vision-language model, MotiMotion reasons about the scene, predicts physically plausible secondary motions, and refines the motion plan before generating the video. For example, if a user draws a trajectory to lift a hand blocking dominoes, the system can infer that the dominoes should begin falling afterward. We also introduce a new benchmark focused on causal physical interactions. Experiments and human studies show that MotiMotion produces videos that are more realistic, physically consistent, and semantically meaningful than previous motion-controlled video generation methods.