HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidance

We present HOI-PAGE, a new approach that prioritizes part-level affordance reasoning to generate high-fidelity 4D human-object interactions (HOIs) from text prompts in a zero-shot fashion. In contrast to prior works that focus on global, whole body-object motion synthesis, our approach explicitly reasons about the underlying part-level mechanics of interactions using large language models (LLMs). We capture this reasoning in a structured part affordance graph (PAG) representation, serving as a high-level interaction scaffolding to guide a three-stage synthesis: first, decomposing input 3D objects into semantic parts; then, generating reference HOI videos from text prompts to extract part-based motion constraints; and finally, optimizing for 4D HOI motion sequences that mimic the reference dynamics while satisfying part-level contact constraints. Extensive experiments show that our approach is flexible and capable of generating complex multi-object or multi-person interaction sequences, with significantly improved realism and text alignment for zero-shot 4D HOI generation.

Animating digital humans interacting with objects, such as ironing a shirt, matters for robot learning, virtual reality, and content creation. But existing methods rely on expensive motion-capture recordings of real humans for supervision, limiting the variety of interactions they can produce. They also treat the human body and object as single units, ignoring which specific body part should touch which object part. We developed HOI-PAGE, a method that generates these interactions directly from a text prompt, with no motion-capture data needed. The key idea is to first ask a large language model to imagine the interaction in detail: which hand grips the iron's handle, where the iron meets the board, and so on. We capture this reasoning as a part affordance graph, which guides the whole generation process: breaking 3D objects into meaningful parts, generating a reference video, and turning that video into the final 4D interaction animation. Because the part affordance graph is flexible, our approach also handles complex scenarios with multiple people or multiple objects, which previous methods struggle with.