Bits That Count: Quantifying and Predicting Capabilities of Language Models

When does learning elicit \textit{existing} knowledge, and when does it primarily teach \textit{new} capabilities? We find that the amount of generalizable information language models learn during training predicts the origins of their emergent capabilities. Minuscule amounts of information---in many cases, a few bits in a single example---can unlock large fractions of models' maximum performance when capabilities are \textit{elicited} rather than \textit{taught}. We quantify these learning regimes using excess description length (EDL), an information-theoretic measure of generalizable information learned during training. We find that elicitation and teaching exhibit distinct EDL signatures that characterize the predominant learning mechanism as information scales: elicitation requires orders of magnitude less information than teaching to comparable performance. We demonstrate that EDL provides a practical tool for quantitatively estimating the maximum amount of predictive information models can compress from data into trainable parameters during learning. These capacity limits describe optimal tradeoffs between data and parameter count that robustly predict when parameter-efficient fine-tuning methods (\textit{e.g.}, LoRA) will underperform full fine-tuning.

When an AI model is fine-tuned on new data and its performance improves, it is not always clear whether the model learned something genuinely new or simply learned to express knowledge it already had from pre-training. This distinction is important, as a model that already "knows" how to do something dangerous but hasn't been prompted correctly poses a very different risk than one that truly lacks that capability. We introduce a way to measure this difference using an information-theoretic quantity called Excess Description Length (EDL), which quantifies how many bits of predictive information a model absorbs during fine-tuning. We find that when models already possess a capability, they need remarkably little information to demonstrate it—sometimes as little as a single training example—and that EDL exhibits distinctive patterns that reliably distinguish this "elicitation" of existing knowledge from "teaching" of new knowledge. We validate these findings across multiple model families, sizes, and task types, including arithmetic, reading comprehension, science questions, instruction following, and complex multi-step reasoning, and we show through controlled experiments that the patterns are causally linked to whether the model has prior relevant knowledge about the task. These measurements can help practitioners assess which capabilities might be easily unlocked in deployed models and design more informed safety evaluations.