Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning

Scaling test-time compute via long Chain-of-Thought unlocks remarkable gains in reasoning capabilities, yet it faces practical limits due to the linear growth of KV cache and quadratic attention complexity. In this paper, we introduce AccordionThinking, an end-to-end framework where LLMs learn to self-regulate the granularity of the reasoning steps through dynamic summarization. This mechanism enables a Fold inference mode, where the model periodically summarizes its thought process and discards former thoughts to reduce dependency on historical tokens. We apply reinforcement learning to incentivize this capability further, uncovering a critical insight: the accuracy gap between the highly efficient Fold mode and the exhaustive Unfold mode progressively narrows and eventually vanishes over the course of training. This phenomenon demonstrates that the model learns to encode essential reasoning information into compact summaries, achieving effective compression of the reasoning context. Our AccordionThinker demonstrates that with learned self-compression, LLMs can tackle complex reasoning tasks with minimal dependency token overhead without compromising solution quality, and it achieves a 3× throughput while maintaining accuracy on a 48GB GPU memory configuration, while the structured step summaries provide a human-readable account of the reasoning process.

Large language models (LLMs) have made remarkable progress in solving complex problems like advanced mathematics and logical reasoning by "thinking step-by-step"—generating long chains of detailed reasoning before reaching a final answer. However, this powerful approach has two critical drawbacks: it becomes increasingly slow and memory-intensive as the reasoning process lengthens, because the model must store every single detail of its previous thoughts. Additionally, these lengthy, unstructured thought processes are often difficult for humans to follow and understand. We introduce "Accordion-Thinking", a new framework that teaches LLMs to think more efficiently and transparently. Inspired by how humans condense complex ideas into concise notes while retaining logical flow, our method trains models to automatically break their reasoning into short segments. After completing each segment, the model writes a brief, precise summary of its key findings, then discards the detailed thought process and continues reasoning using only these summaries. We trained models using this approach and discovered a striking phenomenon: initially, the compressed "folded" reasoning mode (using only summaries) performed worse than the full uncompressed mode. But as training progressed, this performance gap gradually narrowed and eventually vanished entirely. This proves that the models learned to encode all essential reasoning information into their compact summaries, achieving effective compression without losing any critical details. Our Accordion-Thinking models deliver three times faster inference speed on a standard 48GB GPU while maintaining the same accuracy as traditional long-reasoning models. Moreover, the structured step-by-step summaries they generate are highly readable for humans, providing a clear and intuitive explanation of how the model arrived at its final answer.