LLM-Based Rust Code Generation with On-the-Fly Compiler Feedback

Despite recent progress in LLM-based code generation, LLMs still struggle with strongly typed languages such as Rust. The key challenge arises from the complex nature of their type system: it enforces strong formal safety guarantees on the generated code, including powerful semantic properties such as ownership and lifetime. Prior work in enforcing type system constraints explored two general research lines: compiler feedback after code is generated and constrained decoding during generation. Both of these come with practical limitations: the former can incur high latency and delayed feedback, while the latter requires complex compiler reimplementation and white-box access to the LLM. In this work we address this challenge and propose a method for leveraging compiler feedback during code generation: the key idea is to introduce a lightweight transformation that converts partially generated programs into completed ones, enabling immediate on-the-fly compiler feedback during code generation. In turn, this enables low-latency feedback without requiring compiler reimplementation or white box access to the LLM. We implement and extensively evaluate our method on Rust code synthesis and code translation tasks across both frontier and open-weight models. Our experimental results show significantly reduced runtime overhead compared to post-hoc compilation, while preserving and improving compilation rates and functional correctness.