Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generation

Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts with the cognitive process of programming, which is inherently interleaved with forward generation and on-the-fly revision. While prior works attempt to introduce revision via post-hoc agents or external static tools, they either suffer from high latency or fail to leverage the model's intrinsic semantic reasoning. In this paper, we propose Stream of Revision, a paradigm shift that elevates code generation from a monotonic stream to a dynamic, self-correcting trajectory by leveraging the model's intrinsic capabilities. We introduce specific action tokens that enable the model to seamlessly backtrack and edit its own history within a single forward pass. By internalizing the revision loop, our framework Stream of Revision allows the model to activate its latent capabilities just-in-time without external dependencies. Empirical results on secure code generation show that Stream of Revision significantly reduces vulnerabilities with minimal inference overhead.

AI systems that write computer code are now everywhere, but they have a dangerous blind spot: like a writer who can only move forward and never go back to fix a typo, they generate code one piece at a time and cannot revise what they've already written. This means that when the AI accidentally produces an insecure line—the kind of flaw hackers exploit—it is stuck with the mistake and often builds more flawed code on top of it. We taught the AI to catch and repair its own security mistakes on the fly, in the moment it is writing, without any outside help. We gave it special signals it can use to pause, jump back to a risky spot, and patch it—much like how human programmers naturally correct themselves while coding. Our approach makes AI-generated code substantially safer while adding almost no extra time or cost. As AI increasingly writes the software that runs our world, building this kind of self-correction directly into the writing process can help prevent security flaws before they ever reach users.