Continual Model Routing in Evolving Model Hubs

AI model hubs provide access to a rapidly growing collection of powerful pre-trained models, enabling off-the-shelf mixture-of-experts systems with different routing strategies. However, this rapid growth poses two fundamental challenges: scaling model selection across thousands of experts and continually updating routing mechanisms as new models and tasks are introduced. In this paper, we formalise this setting as Continual Model Routing (CMR) and propose CMRBench, a new large-scale benchmark simulating realistic hub expansion and including over 2,000 candidate models. Finally, we introduce CARvE, a contrastive embedding approach for efficient continual model routing via domain-stratified coreset replay and checkpoint-based anchoring. Extensive empirical results and ablations show that CARvE significantly outperforms zero-shot retrieval, fine-tuning, and adapter-merging baselines in model, family, and domain-level accuracy.

Today we no longer ask whether a model can be built, but which model, among many thousands already available, should answer a given question. These collections are not fixed. New models arrive, others fall out of use, and so the act of choosing well cannot be settled once and then forgotten. We argue that this choice is itself a form of learning that must continue over time. We give the problem a name, Continual Model Routing, and a benchmark on which it can be studied honestly. We then propose a method that learns to point each request toward a suitable model by comparing meanings rather than running candidates, while taking care not to lose what it already knew as the collection grows. The result is a selector that stays accurate, stable, and affordable as the world of models keeps changing.