ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

Large reasoning models (LRMs) typically solve reasoning-intensive tasks by generating long chain-of-thought (CoT) traces, leading to substantial inference overhead. We identify a reproducible inference-time phenomenon, termed \textbf{\emph{Self-Compression}}: when multiple independent and answerable questions are presented within a single prompt, the model spontaneously produces shorter reasoning traces for each question. This phenomenon arises from \emph{multi-question contextual pressure} during generation and consistently manifests across models and benchmarks. Building on this observation, we propose ConPress (Learning from Contextual Pressure, a lightweight self-supervised fine-tuning approach. ConPress constructs multi-question prompts to induce self-compression, samples the resulting model outputs, and parses and filters per-question traces to obtain concise yet correct reasoning trajectories. These trajectories are directly used for supervised fine-tuning, internalizing compressed reasoning behavior in single-question settings without external teachers, manual pruning, or reinforcement learning. With only 8k fine-tuning examples, ConPress reduces reasoning token usage by 59\% on MATH500 and 33\% on AIME25, while maintaining competitive accuracy.

Large reasoning models can solve difficult problems, but they often produce very long step-by-step solutions, making them slower and more expensive to use. We find a simple but surprising behavior: when a model answers several independent questions in one prompt, it naturally gives shorter reasoning for each question while keeping the key steps. We call this self-compression. Based on this finding, we propose ConPress, a simple training method that teaches the model to keep this concise reasoning style for normal single-question use. ConPress uses the model’s own correct and shortened answers as training data, without extra teacher models, human-written shorter solutions, or complex reinforcement learning. Experiments show that ConPress greatly reduces generated tokens while largely preserving accuracy, making reasoning models more efficient and practical.