CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM

Users increasingly face the challenge of selecting an appropriate LLM for a given task from a rapidly growing pool of LLMs, each with distinct but often opaque latent properties. Compounding this challenge, users may lack the vocabulary or awareness to explicitly articulate the characteristics they value in an LLM's responses or deployment. We propose an interaction-efficient active learning framework in which a dueling bandit algorithm iteratively selects pairs of LLMs, collects user feedback about their responses, and updates its belief about the user's latent preferences. We introduce a novel belief-aware upper confidence bound strategy that balances exploration of the model pool with exploitation of inferred preferences, enabling efficient alignment between user needs and LLM capabilities under user-specified cost and time budgets. Through diverse experiments on LLMs and human studies, we experimentally verify that our model can efficiently match users to LLMs at a lower cost.

Large machine learning models, ranging from LLMs to Agentic AI and diffusion models, are being created and deployed at an increasingly rapid pace, creating difficulties for users to select a suitable model for their task of interest, even when they know certain properties of the models (e.g. cost, reasoning capabilities, latency, etc). We argue that this challenge arises in part because users often possess preferences that are difficult to explicitly articulate or systematically map to model capabilities, such as preferences for explanation style, conversational behavior, creativity, factual conservatism, or multimodal interaction patterns. Our paper presents a framework that repeatedly proposes two models from a large pool of models for the user to select and provide an optional feedback, given a cost and time budget. From the user's choice and feedback, our framework selects the next pair of models. Empirically, our framework demonstrates efficiency in cost and competitive performance to our baseline method, especially in more abstract scenarios, through diverse automated and real-world experiments.