Iterated Population Based Training with Task-Agnostic Restarts

Hyperparameter Optimization (HPO) can lift the burden of tuning hyperparameters (HPs) of neural networks. HPO algorithms from the Population Based Training (PBT) family are efficient thanks to dynamically adjusting HPs every few steps of the weight optimization. Recent results indicate that the number of steps between HP updates is an important meta-HP of all PBT variants that can substantially affect their performance. Yet, no method or intuition is available for efficiently setting its value. We introduce Iterated Population Based Training (IPBT), a novel PBT variant that automatically adjusts this HP via restarts that reuse weight information in a task-agnostic way and leverage time-varying Bayesian optimization to reinitialize HPs. Evaluation on 8 image classification and reinforcement learning tasks shows that, on average, our algorithm matches or outperforms 5 previous PBT variants and other HPO algorithms (random search, ASHA, SMAC3), without requiring a budget increase or any changes to its HPs. The source code is available online.

When teaching modern AI systems to solve a problem, researchers need to specify many settings, such as how strongly each update should change the AI. To avoid manual trial-and-error tuning of such settings, researchers use automatic tuning methods, which unfortunately often require manual tuning themselves. One efficient way to automatically tune settings is by simultaneously updating the AI and the settings of the updating process. However, existing methods that work like this require manual specification of how often the settings should be changed. Despite the importance of this decision for final results, there are no guidelines on how to make it. We introduce a new tuning method that automatically adapts how often the settings are changed while the AI is learning. When progress slows down, our method partially restarts while keeping useful information from what was already learned (which helps it efficiently explore different settings of the AI learning process). We evaluate our method on tasks involving image recognition and control of a robotic body in computer simulations. Our method matches or outperforms eight existing methods without requiring extra budget or manual adjustments. This could make teaching AI systems to solve problems more convenient and efficient.