Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement

Autoregressive language models frequently degrade during long-horizon generation, producing repetitive text, losing instruction adherence, and exhibiting unstable entropy. Despite the prevalence of these failures, practitioners lack online diagnostics to detect them in real-time as they occur. We formalize this degradation as cognitive fatigue, a measurable generation-time state characterized by decay in attention to the original prompt, representational drift, and entropy miscalibration. We introduce the Fatigue Index (FI), a lightweight, model-agnostic diagnostic that aggregates these three signals under explicit axioms (monotonicity, boundedness, interpretability) enabling reliable runtime monitoring. Across nine models (1B–13B parameters), FI trajectories exhibit structured temporal dynamics, predict task degradation (AUROC = 0.95) and repetition ($\rho = 0.94$), and reveal non-monotonic scaling behavior: instruction-tuned models below 3B exhibit faster collapse than base models, with this trend reversing at 7B. Stress analyses further show that FI onset accelerates under longer contexts, middle-positioned evidence, and reduced numerical precision. These results establish cognitive fatigue as a coherent and measurable phenomenon, and position FI as a principled tool for runtime reliability monitoring in production LLM systems.

Large language models (LLMs) like ChatGPT can struggle when asked to generate long responses. They may start repeating themselves, forget the original instructions, or produce increasingly erratic output. This happens gradually during generation, but until now there was no way to detect it in real time as it was occurring. We call this phenomenon cognitive fatigue, borrowing the term as an analogy for how a model progressively "loses focus" over a long generation. We introduce a simple, automatic measurement called the Fatigue Index (FI) that tracks three warning signs simultaneously: whether the model is still paying attention to the original prompt, whether its internal representations are drifting, and whether its output distribution is becoming unstable. FI runs alongside the model during generation without requiring any retraining or changes to the model. We tested it across nine models of varying sizes and showed it reliably predicts when output quality is about to deteriorate. We also found that models fine-tuned to follow instructions can actually become more fragile than their base counterparts at smaller sizes, a counterintuitive finding we call the "alignment tax on cognitive stamina." FI is designed to be a practical monitoring tool for anyone deploying LLMs in real-world settings where long, reliable generation matters.