Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging

Model merging, which combines multiple models into a single model, has gained popularity in recent years. By efficiently integrating the capabilities of various models, this significantly reduces the parameter count and memory usage. However, current methods can only produce one single merged model. This necessitates a performance trade-off due to conflicts among the various models, and the resultant one-size-fits-all model may not align with the preferences of different users who may prioritize certain models over others. To address this issue, we propose preference-aware model merging, and formulate this as a multi-objective optimization problem in which the performance of the merged model on each base model's task is treated as an objective. In a single merging process, the proposed parameter-efficient structure generates a Pareto set of merged models, with each representing a Pareto-optimal solution for a preference. Users can then select merged models tailored to their preferences from this learned Pareto set. Experimental results demonstrate that the proposed Pareto Merging produces diverse trade-off models and achieves higher test accuracy compared to state-of-the-art merging baselines.

Combining multiple deep learning models into one is a popular way to save computing resources and memory. The challenge is that current methods typically produce a single, "one-size-fits-all" merged model. This often means sacrificing performance on some tasks, and the final model may not align with what different users actually need, especially if they prioritize certain original models' abilities.To address this, we introduce "Preference-Aware Model Merging." Our approach doesn't just create one merged model; instead, it efficiently generates a collection of diverse merged models in a single process. Each model in this collection offers a unique, optimal trade-off, reflecting different possible user preferences for the strengths of the original AIs.This allows users to select a merged model from this collection that is best suited to their specific requirements. Our experiments demonstrate that this method provides a wider variety of effective models and achieves better overall performance compared to existing model merging techniques.