Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deployment

Due to limited resources and public safety concerns, deep reinforcement learning (RL) agents for many cyber-physical systems (e.g., autonomous vehicles) are first trained in simulators. However, when deployed in real world environments, they often suffer from performance degradation or safety violations because of the inevitable Sim2Real gap. Existing zero-shot approaches, such as robust safe RL and domain randomization, mitigate this issue but typically at the cost of degraded performance or residual safety risks when experiencing unmodeled system dynamics. To address these limitations, we propose a novel reinforcement learning framework that enables safe and efficient policy transfer via probabilistic latent embeddings and dynamic policy adaptation. We consider a family of Constrained Markov Decision Processes (CMDPs) under different environment contexts. By leveraging latent context variable in meta-RL, the proposed framework infers the latent representation of the environment from simulated experiences. Furthermore, it incorporates a distributional RL formulation, which allows risk levels of the deployed policy to be adjusted dynamically, based on the estimation accuracy of the latent context variable. This strategy promotes safety at the early deployment stage and improves efficiency through fast policy adaptation under the Sim2Real gap.

Reinforcement learning policies for cyber-physical systems such as autonomous vehicles are often trained in simulators due to limited resources and public safety concerns. However, when deployed in real-world environments, these policies may suffer from performance degradation or safety violations because of the inevitable Sim2Real gap. We propose a reinforcement learning framework for safe and efficient policy transfer via probabilistic latent embeddings and dynamic policy adaptation. The framework uses a latent context variable to infer the environment context and adjusts the deployed policy’s risk level based on the estimation accuracy of this latent variable. This strategy promotes safety during early deployment and improves efficiency through fast policy adaptation under the Sim2Real gap.