MaMi-HOI: Harmonizing Global Kinematics and Local Geometry for Human-Object Interaction Generation

Generating realistic 3D Human-Object Interactions (HOI) is a fundamental task for applications ranging from embodied AI to virtual content creation, which requires harmonizing high-level semantic intent with strict low-level physical constraints. Existing methods excel at semantic alignment, however, they struggle to maintain precise object contact. We reveal a key finding termed $\textit{Geometric Forgetting}$: as diffusion model depth increases, semantic feature tend to overshadow object geometry feature, causing the model to lose its perception to object geometry. To address this, we propose MaMi-HOI, a hierarchical framework reconciling Macro-level kinematic fluidity with Micro-level spatial precision. First, to counteract geometric forgetting, we introduce the Geometry-Aware Proximity Adapter (GAPA), which explicitly re-injects dense object details to perform residual snapping corrections for precise contact. Nevertheless, such aggressive local enforcement can disrupt global dynamics, leading to robotic stiffness. In response, we introduce the Kinematic Harmony Adapter (KHA), which proactively aligns whole-body posture with spatial objectives, ensuring the skeleton actively accommodates constraints without compromising naturalness. Extensive experiments validate that MaMi-HOI simultaneously achieves natural motion and precise contact. Crucially, it extends generation capabilities to long-term tasks with complex trajectories, effectively bridging the gap between global navigation and high-fidelity manipulation in 3D scenes.

Digital characters are increasingly used in games, virtual reality, robotics, and embodied AI, but making them interact naturally with objects is still difficult. A person should not only appear to understand an instruction such as “sit on the chair” or “pick up the box”; their body must also touch the object in the right places, at the right time, without looking stiff or unnatural. We found that current AI systems for generating human motion often focus on the overall meaning of an action while gradually losing awareness of the object’s exact shape and position. This can produce motions that look reasonable at first glance but fail at important physical details, such as a hand missing a handle or a body floating above a seat. We propose MaMi-HOI, a method that helps generated human motions balance two needs at once: smooth whole-body movement and accurate contact with nearby objects. It first restores fine object details when precise contact is needed, and then adjusts the full body so the motion remains natural rather than forced. This allows digital humans to move through 3D scenes, follow longer action plans, and interact with objects more reliably. Our work can help create more believable virtual characters and support future systems that need to understand and act in physical environments.