LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models

Despite the strong performance of Large Language Models (LLMs) on complex instruction-following tasks, precise control of output length remains a persistent challenge. Existing methods primarily attempt to enforce length constraints by externally imposing length signals or optimization objectives, while largely overlooking the underlying limitation: the model's intrinsic deficit in length cognition. To address this, we propose LARFT (Length-Aware Reinforcement Fine-Tuning), a training framework that aligns the model's length cognition with its action. Specifically, LARFT integrates length-oriented reinforcement learning with a hindsight length awareness. By transforming on-policy data into hindsight self-awareness tasks where the model learns to identify the actual length of its own generation, LARFT jointly optimizes the model’s internal representation of length information and refines its policy to satisfy length constraints, thereby achieving precise and reliable length instruction following. Extensive experiments across four base models demonstrate that LARFT outperforms existing baselines, achieving an average improvement of +20.92 points across three length instruction following benchmarks with only a marginal decline of -1.45 points on four general capability benchmarks. Our code is available at https://github.com/Captain-zhangw/LARFT.

Modern AI models are capable of complex reasoning, yet they consistently struggle with a seemingly simple task: producing text of an exact length. When asked for a 100-word summary, they often write 150 words because they lack an internal sense of how much they have written—akin to a speaker who talks without watching the clock. To solve this, we developed a training method that teaches the AI to become self-aware of its own output length. Instead of simply penalizing the AI for missing a word count, we train it to reflect on and accurately guess the length of the text it just produced, bridging the gap between what the AI "knows" and what it "does." Our experiments show this self-awareness approach drastically improves the AI's ability to follow strict length instructions without sacrificing its general intelligence, making AI assistants much more reliable for tasks requiring precise formatting.