Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks. However, task-level expert selection is often too coarse-grained, since different instances may require different expertise. To address this, we propose Skill-MoE, a symbolic, skill-based, and gradient-free Mixture-of-Experts framework for instance-level expert selection. Skill-MoE infers skills (e.g., algebra in mathematics) from each query, selects experts based on skill relevance, and lets each expert generate its own reasoning. The resulting k outputs are then synthesized by an aggregator chosen for its ability to integrate diverse responses. While instance-level selection substantially improves performance, naively implementing it incurs heavy overhead from repeated model loading and offloading. We address this with a batch inference strategy that groups instances by assigned experts, allowing each model to be loaded only once. As a result, Skill-MoE integrates 16 expert models on a single GPU with runtime comparable to prior multi-agent baselines using 4 GPUs. Across diverse benchmarks (MMLU-Pro, GPQA, AIME, and MedMCQA), Skill-MoE achieves an average absolute improvement of 8.15% over the best baseline. It also generalizes well to unseen tasks and outperforms discussion-based methods without requiring expensive multi-round interactions.

Large language models (LLMs) often excel at different kinds of reasoning tasks, such as solving math problems, answering scientific questions, or handling medical knowledge. However, choosing a single "best" model for an entire task can be too simplistic, because different questions may require different types of expertise. We introduce Skill-MoE, a system that automatically identifies the skills needed for each individual question (e.g., algebra for math problems) and selects the most relevant expert models to answer it. Each selected model independently generates its reasoning, and a final aggregator model combines these responses into a stronger answer. Unlike many previous multi-agent approaches, our method does not require expensive multi-round discussions between models but still delivers strong performance. A major challenge of this fine-grained, instance-level expert selection is computational cost, since repeatedly loading many large models can be very slow. We address this with a batching strategy that efficiently groups questions by expert usage, allowing 16 expert models to run on a single GPU with practical runtime. Across a wide range of reasoning benchmarks, Skill-MoE substantially improves accuracy and generalizes well to new tasks, showing that inferring skills to combine expert models can produce stronger reasoning.