HeroWorld: Long-Horizon Action-Conditioned World Models for Third-Person Games

Interactive world models turn video generation models into controllable simulators, but most game-based systems are developed in first-person settings where actions are expressed mainly through camera motion. Third-person control is more challenging because the model must jointly synthesize scene evolution, camera movement, visible character dynamics, and character-environment interaction. We introduce HeroWorld, a third-person action-conditioned world model adapted from a pretrained game video generator using curated human gameplay videos and agent-generated Minecraft rollouts. HeroWorld uses bidirectional finetuning, causal finetuning, and progressive horizon distillation, which gradually expands autoregressive rollout length to mitigate the train-short--test-long mismatch and enable stable 30-second controllable Zelda rollouts. We further study out-of-distribution transfer across unseen scenes, characters, and game domains, finding that robustness depends on preserving character-background separation and action-dependent interaction with surrounding geometry. Action-vector scaling, attention injection, and multi-game training improve robustness under domain shifts, offering practical guidance for building reliable third-person world models from pretrained video generators.