MoSA: Motion-constrained Stress Adaptation for Mitigating Real-to-Sim Gap in Continuum Dynamics via Learning Residual Anisotropy

Learning real-world dynamics from visual observations is crucial for various domains. A common strategy is to calibrate simulators by estimating physical parameters, yet accuracy is ultimately bounded by the underlying physical models, which often assume materials are homogeneous and isotropic. Even if reasonable, real-world objects typically exhibit mild anisotropy and heterogeneity. After the near-isotropic backbone is well calibrated, these residual effects become the key bottleneck for further closing the real-to-sim gap. Although neural networks can fit dynamics end-to-end, such black-box modeling discards strong physical priors, leading to poor data efficiency and overfitting. Therefore, we propose MoSA, a motion-constrained stress adaptation framework that targets these residual effects to further improve real-to-sim dynamics learning. MoSA uses an isotropic model as a physics prior and learns residual stress operators to capture mild anisotropy and heterogeneity. It progressively adapts stresses via microplane-constrained redistribution in a physics-informed cascaded network. We further impose motion constraints by supervising temporal and spatial derivatives of the deformation field. Experimentally, our learned dynamics achieves superior accuracy, generalization, and robustness, while learning physically meaningful residual anisotropy. Finally, we validate MoSA in a robot manipulation setting, showing that better real-to-sim dynamics modeling translates into more reliable sim-to-real transfer.

Accurately simulating how real objects bend, stretch, or change shape is crucial for fields like computer graphics, robotics, and digital twins. However, real-world objects are usually much more complex than the simplified material models used in software simulators. Even if an object looks pretty uniform on the outside, it might have tiny variations on the inside, or it might react differently to forces depending on the direction. Over time, these seemingly small details can cause a simulation to drift away from how the object would actually move in real life. To solve this, this paper introduces MoSA, a video-based method designed to improve how we simulate real objects. Instead of tossing out the laws of physics and replacing them with a massive "black box" AI model, MoSA starts with a standard physics simulator. It simply learns to fill in the missing tweaks and corrections needed to make the simulation match the real thing. It watches videos of an object deforming to understand its movement over time, and then fine-tunes the internal forces within the simulation so that the virtual motion closely mirrors reality. Tests using both computer-generated data and real objects show that MoSA creates more accurate and stable simulations compared to older methods. The adjustments the model learns actually make physical sense, too—for instance, they can map out how some specific parts inside an object are stiffer than others. Finally, the paper shows that having these highly accurate simulations can help robots handle bendable or squishy objects much more reliably in the real world.