The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level

Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While MoE architectures are primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we _zoom out_ from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., closing brackets in LaTeX). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability. Code is available at: https://github.com/jerryy33/MoE_analysis.

Modern AI language models are so complex that even their creators struggle to understand what is happening inside them, which is a serious obstacle to making these systems safe and trustworthy. One promising design, called Mixture-of-Experts (MoE), splits a model's work across many modules, using only a small fraction for any given input. We investigated whether this design makes models easier to understand, or whether the inner workings remain just as opaque. We tested twelve models and found that individual units inside MoE designs tend to respond to one concept at a time, rather than many jumbled concepts simultaneously. Crucially, the fewer modules used at once, the cleaner this effect becomes. Building on this, we treated each module as a single interpretable unit and automatically generated plain-language descriptions for hundreds of them, then verified those descriptions were meaningful and not just plausible. Our analysis also settles a long-running debate: these modules are neither broad topic specialists (e.g., biology) nor simple grammar processors. Instead, they perform precise tasks, like closing brackets in mathematical notation or completing role-playing game terminology. This suggests that the industry trend toward using fewer modules at once is making AI more transparent by design.