Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance

Multi-objective learning under user-specified preference is common in real-world problems such as multi-lingual speech recognition under fairness. In this work, we frame such a problem as a semivectorial bilevel optimization problem, whose goal is to optimize a pre-defined preference function, subject to the constraint that the model parameters are weakly Pareto optimal. To solve this problem, we convert the multi-objective constraints to a single-objective constraint through a merit function with an easy-to-evaluate gradient, and then, we use a penalty-based reformulation of the bilevel optimization problem. We theoretically establish the properties of the merit function, and the relations of solutions for the penalty reformulation and the constrained formulation. Then we propose algorithms to solve the reformulated single-level problem, and establish its convergence guarantees. We test the method on various synthetic and real-world problems. The results demonstrate the effectiveness of the proposed method in finding preference-guided optimal solutions to the multi-objective problem.

This paper addresses how to train machine learning models when there are multiple goals to achieve and the user has a specific preference for how to balance them. Instead of treating preferences as fixed weights or strict constraints, which have theoretical and practical limitations, we treat them as an extra goal to optimize -- over models that already do well across all objectives. This creates a layered decision-making problem, where we aim to find the best option based on preference, but only among solutions that already perform well on the main objectives. To simplify this nonsmooth nonconvex layered problem, we use a penalty method and a smooth approximation to the original problem. We establish the theoretical relation between the solutions of the reformulated problem and the original one. Interestingly and perhaps surprisingly, even though the stationary solutions to such problems often need second-order information, we show that first-order approaches can still be used to approximate such solutions. This serves as a foundation for developing efficient first-order algorithms to solve the problem with convergence guarantees. Experiments on real-world applications such as fairness-aware multi-lingual speech recognition show that our method is both practical and effective.