Implicit Action Chunking for Smooth Continuous Control

Reinforcement learning often produces high-frequency oscillatory control signals that undermine the safety and stability required for physical deployment. Explicit action chunking addresses this by predicting fixed-horizon trajectories but scales the policy output dimension proportionally with the horizon length, leading to optimization difficulties and incompatibility with standard step-wise interaction. To overcome these challenges, this paper proposes Dual-Window Smoothing (DWS), an implicit action chunking framework for smooth continuous control. Unlike explicit methods, DWS enforces temporal coherence without expanding the action space. It uses a dual-window design: an execution window that ensures physical smoothness through deterministic modulation, and a value window that aligns temporal-difference targets over the horizon to correct critic bias caused by open-loop execution. DWS also includes a lightweight actor-side temporal regularizer based on first-order action differences to promote global continuity. This design effectively bridges the gap between temporal abstraction and reactive step-wise control. Experiments on benchmarks including the DeepMind Control Suite and industrial energy management tasks show that DWS outperforms state-of-the-art (SOTA) baselines. In complex vision-based autonomous driving tasks, DWS achieves smoother control, safer behavior with reduced jitter, and attains a 100% success rate.

Reinforcement learning has shown impressive results in robotics, autonomous driving, and energy systems. However, many learned controllers produce rapid and unstable control signals, such as sudden steering corrections or power fluctuations, which are difficult and unsafe to apply in real-world physical systems. This limits the deployment of reinforcement learning in safety-critical applications. This paper introduces a new method called Dual-Window Smoothing (DWS) to make reinforcement learning controllers behave more smoothly and reliably. Instead of generating abrupt actions at every instant, DWS encourages the controller to produce temporally consistent decisions while still remaining responsive to changing environments. Importantly, the method achieves this without increasing the complexity of the control system or slowing down decision making. The proposed approach is tested on a wide range of tasks, including robotic control, electric vehicle energy management, and autonomous driving. Across these experiments, DWS consistently produces smoother and safer behavior while also improving overall task performance. In challenging autonomous driving scenarios, the method achieves stable and human-like control and reaches a 100% success rate in safety-critical overtaking tasks. These results demonstrate that improving temporal consistency is essential for deploying reinforcement learning in real-world systems. The proposed framework provides a practical way to make AI-based control safer, smoother, and more reliable for future intelligent machines and vehicles.