ABO-Med: Accelerated Bilevel Optimization for Few-Shot Medical Image Classification

In recent years, bilevel optimization has been widely used in a variety of machine learning tasks. However, prior bilevel optimization algorithms generally require the computation of second-order information, which limits their practical scalability. Only recently has a first-order paradigm for bilevel optimization been established, attaining near-optimal theoretical guarantees for solving bilevel optimization problems. In this paper, we propose ABO-Med, a scalable instantiation of this paradigm for few-shot learning, by incorporating it into the model-agnostic meta-learning (MAML) framework and tailoring it to medical image classification. Theoretically, ABO-Med establishes the optimality of MAML-type meta-learning approaches. Empirically, it generally outperforms prior baselines on several public medical datasets, achieving performance gains of around 1\% to 21.4\% over previous state-of-the-art methods.