MCCE: A Framework for Multi-LLM Collaborative Search in Discrete Spaces with Similarity-Filtered Preference Learning

Multi-objective discrete optimization problems, such as molecular design, pose significant challenges due to their vast and unstructured combinatorial spaces. Traditional evolutionary algorithms often get trapped in local optima, while expert knowledge can provide crucial guidance for accelerating convergence. Large language models (LLMs) offer powerful priors and reasoning ability, making them natural optimizers when expert knowledge matters. However, closed-source LLMs, though strong in exploration, cannot update their parameters and thus cannot internalize experience. Conversely, smaller open models can be continually fine-tuned but lack broad knowledge and reasoning strength. We introduce Multi-LLM Collaborative Co-evolution (MCCE), a hybrid framework that unites a frozen closed-source LLM with a lightweight trainable model. The system maintains a trajectory memory of past search processes; the small model is progressively refined via reinforcement learning, with the two models jointly supporting and complementing each other in global exploration. Unlike model distillation, this process enhances the capabilities of both models through mutual inspiration. Experiments on multi-objective drug design benchmarks show that MCCE achieves state-of-the-art Pareto front quality and consistently outperforms baselines. These results highlight a new paradigm for enabling continual evolution in hybrid LLM systems, combining knowledge-driven exploration with experience-driven learning. The code of MCCE is available on https://github.com/lzz-z/MCCE

Large language models (LLMs) are becoming promising tools for solving difficult optimization problems, such as molecular design and combinatorial search. However, current approaches typically rely on a single frozen LLM. While these models have strong reasoning ability, they cannot truly learn from past optimization experience through parameter updates. As a result, they often lose diversity over time and repeatedly explore similar solutions. Retrieval-based methods can inject external knowledge, but they still cannot continuously internalize experience. In this work, we introduce MCCE, a collaborative co-evolution framework that combines a powerful closed-source LLM with a lightweight trainable local model. The large model performs broad global exploration, while the smaller model continuously learns from successful search trajectories and adapts over time, and provides strong candidates based on experience to help the close-source LLM. The two models iteratively improve each other through a shared feedback loop. We demonstrate that this collaborative learning paradigm achieves state-of-the-art performance on multi-objective molecular optimization tasks. Beyond drug discovery, the framework also generalizes to other combinatorial optimization problems, suggesting a broader direction for continual learning and collaborative AI optimization systems.