HyMTRL: A Hybrid Multi-Task Reinforcement Learning Framework via Phased Policy Evolution

Multi-task reinforcement learning (MTRL) aims to improve sample efficiency by sharing knowledge across related tasks, but it often suffers from asynchronous convergence speed caused by inherent differences in task difficulty. This imbalance places substantial representational strain on the shared critic network, which emerges as a major performance bottleneck. To address this issue, we propose Hybrid Multi-Task Reinforcement Learning (HyMTRL), a framework that alleviates critic overload through a phased policy evolution strategy. HyMTRL divides task learning into a reinforcement exploration phase and an imitation refinement phase. By transitioning mastered tasks from reinforcement learning–based policy optimization to imitation learning–based behavior consolidation, these tasks are removed from the critic’s optimization objective, effectively reducing representational strain. In addition, a critic reset mechanism restores network capacity while preserving learned policy and historical experience. HyMTRL is a general framework that can be easily integrated with a wide range of existing MTRL methods. Empirical evaluations on the MetaWorld benchmark demonstrate that HyMTRL leads to significant improvements in both learning efficiency and final performance.

When training Artificial Intelligence (AI) to perform multiple tasks simultaneously, the goal is to improve efficiency by allowing the AI to share knowledge across tasks. However, because tasks vary in difficulty, the AI learns them at different speeds. Juggling easy and hard tasks at the same time overwhelms the evaluative part of the AI’s "brain," creating a bottleneck that slows down the entire learning process. To solve this, we developed a new training framework called HyMTRL. Instead of forcing the AI to continuously evaluate every task, our method uses a two-step approach. First, the AI learns through active exploration. Once it masters a task, it stops actively trying to "solve" it and instead simply memorizes its successful behavior. By taking mastered tasks off the AI's plate, we free up its processing power to focus entirely on the harder, unfinished tasks. Combined with a "refresh" mechanism that clears out system clutter without erasing learned skills, our method significantly boosts both how fast the AI learns and how well it performs.