Breaking Multi-Task Curse: Reward-Weighted Evolution for Black-Box Many-Task Optimization

Evolutionary multi-tasking accelerates black-box optimization via knowledge transfer but falters in scenarios involving many low-similarity tasks. We identify this scalability barrier as the Multi-Task Curse, driven by evaluation budget dispersion and negative transfer. To overcome this, we propose MES-RET (Many-task Evolution Strategy with Reward-weighted Evaluation and Transfer), which combats budget dispersion via a reward-weighted evaluation scheme that guarantees superior expected improvement, while simultaneously mitigating negative transfer through a robust reward-weighted aggregation of mean and covariance statistics, ensuring a safe fallback to independent evolution. Furthermore, to handle neural dimensional mismatches in many-task policy search, we introduce a semantic parameter alignment strategy that bridges heterogeneous state-action spaces. Extensive experiments on synthetic benchmarks, real-world engineering problems, and reinforcement learning tasks demonstrate that MES-RET consistently outperforms state-of-the-art methods, notably enabling skill transfer across morphologically distinct policies.

Using nature-inspired evolution to optimize artificial intelligence (AI) for multiple problems simultaneously is usually highly efficient because the AI can share successful traits across tasks. However, when evolving solutions for dozens of very different problems at once, the system often gets overwhelmed—it spreads its computing time too thin and mistakenly shares unhelpful traits between unrelated tasks, making it perform worse than if it tackled each problem independently. We call this the Multi-Task Curse. To break this curse, we developed a new evolution strategy called MES-RET. Instead of treating all tasks equally, our method dynamically focuses its computing power on the tasks showing the most evolutionary promise. It also uses a safety valve to carefully filter shared knowledge, instantly rejecting bad genetic advice to prevent confusion. Furthermore, we designed a way for AI agents controlling differently shaped robots to still share movement skills. Our extensive tests on complex math puzzles, real-world engineering designs, and virtual robot simulations prove that our method successfully overcomes the curse, evolving high-quality solutions faster and more reliably than existing approaches.