Uncovering Competency Gaps in Large Language Models and Their Benchmarks

The evaluation of large language models relies heavily on standardized benchmarks. These benchmarks provide useful aggregated metrics, but can obscure (i) particular sub-areas where the models are weak ("model gaps") and (ii) imbalanced coverage in the benchmarks themselves ("benchmark gaps"). To automatically uncover both types of gaps, we propose a simple new method using concept activations from sparse autoencoders, to identify fine-grained gaps on a per-concept basis. The method also benefits from grounding evaluation in the model's internal representations, as well as easy comparison across benchmarks. We applied the method to five popular open-source models and more than a dozen benchmarks, as illustrative examples. As validation of the approach, we found that our automatic, unsupervised method was able to recover model gaps that have been previously documented in the literature (e.g. relating to sycophancy), in addition to identifying novel model gaps. We were also able to automatically uncover benchmark gaps: core concepts that should fall within the scope of a given benchmark. Our "competency gaps" method can be used to complement existing benchmarks, by providing a concept-level decomposition of model behavior, and by helping benchmark developers iterate upon benchmark design. Code is available at https://competency-gaps.github.io.

We test AI language models the same way we test students: with standardized exams, usually referred to as benchmarks. A model gets a score, and we move on. But a single score can hide a lot. A model might be doing great overall while quietly failing on a specific kind of question, and we'd never know. Worse yet, the exam itself might skip topics it was supposed to cover. We built a simple, automatic way to look past the score and see how the model performs, concept by concept. Instead of relying on humans to guess what to check, our method reads the patterns inside the model itself to find these blind spots. It also flags gaps in the tests: important topics a benchmark claims to measure but doesn't really contain. When we tried it on five popular open models across more than a dozen benchmarks, it rediscovered weaknesses researchers already knew about (like a tendency to be sycophantic, i.e., just agree with the user), and surfaced new ones nobody had flagged. We see this as a tool to sit alongside existing benchmarks, helping people understand model behavior in finer detail, and helping benchmark builders see what their tests are missing.