The Appeal and Reality of Recycling LoRAs with Adaptive Merging

The widespread availability of fine-tuned LoRA modules for open pre-trained models has led to an interest in methods that can adaptively merge LoRAs to improve performance. These methods typically include some way of selecting LoRAs from a pool and tune merging coefficients based on a task-specific dataset. While adaptive merging methods have demonstrated improvements in some settings, no past work has attempted to recycle LoRAs found ``in the wild'' on model repositories like the Hugging Face Hub. To address this gap, we consider recycling from a pool of nearly 1,000 user-contributed LoRAs trained from the Llama 3.1 8B-Instruct language model. Our empirical study includes a range of adaptive and non-adaptive merging methods in addition to a new method designed via a wide search over the methodological design space. We demonstrate that adaptive merging methods can improve performance over the base model but provide limited benefit over training a new LoRA on the same data used to set merging coefficients. We additionally find not only that the specific choice of LoRAs to merge has little importance, but that using LoRAs with randomly initialized parameter values yields similar performance. To better understand why past work has proven successful, we confirm that positive transfer is indeed possible when there are highly relevant LoRAs in the pool. We release the model checkpoints and code online at https://github.com/r-three/realistic-adaptive-merging.

Modern large-language models are generally good at many tasks but not all; they can be further specialized for new task domains using small add-ons (like skill patches). Individual contributors have developed and shared thousands of these patches, freely available for public use. Merging these publicly available skill patches into a single model to enhance specialization in a particular task has therefore emerged as an exciting prospect. We test the viability of this idea by recycling nearly 1,000 skill patches that individual contributors have uploaded to public repositories. Unfortunately, our work shows that merging these patches is not as effective as training a single fresh skill patch on the same task’s data, which is a simple and straightforward alternative. We find that merging these patches is successful only when the pool consists of highly relevant patches, which is unrealistic in a practical recycling setting. Our analysis suggests that prior work’s successes with skill patch merging may not stem from combining expertise across patches, but rather from a side effect that incidentally stabilizes model behavior. Overall, our finding indicates that effortlessly recycling skill patches is more complicated than it appears.