Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making

Offline decision-making via diffusion models often produces trajectories that are misaligned with system dynamics, limiting their reliability for control. We propose Model Predictive Diffuser (MPDiffuser), a compositional diffusion framework that combines a diffusion planner with a dynamics diffusion model to generate task-aligned and dynamically plausible trajectories. MPDiffuser interleaves planner and dynamics updates during sampling, progressively correcting feasibility while preserving task intent. A lightweight ranking module then selects trajectories that best satisfy task objectives. The compositional design improves sample efficiency and adaptability by enabling the dynamics model to leverage diverse and previously unseen data independently of the planner. Empirically, we demonstrate consistent improvements over prior diffusion-based methods on unconstrained (D4RL) and constrained (DSRL) benchmarks, and validate practicality through deployment on a real quadrupedal robot.

MPDiffuser is a diffusion-based framework for offline decision making that generates trajectories which are both task-oriented and consistent with system dynamics. Instead of relying on a single diffusion model, the method combines a planner model that proposes high-reward behaviors with a dynamics model that refines trajectories to better satisfy the underlying physics of the environment. By alternating between these two models during sampling, MPDiffuser produces trajectories that are more reliable for control while retaining the flexibility and expressiveness of diffusion-based planning. The framework also supports safety constraints and adaptation to new dynamics, and demonstrates strong performance across offline decision-making benchmarks and real-world robot locomotion tasks.