Pull Requests as a Training Signal for Repo-Level Code Editing

Repository-level code editing requires models to understand complex dependencies and execute precise multi-file modifications across a large codebase. While recent gains on SWE-bench rely heavily on complex agent scaffolding, it remains unclear how much of this capability can be internalised via high-quality training signals. To address this, we propose Clean Pull Request (Clean-PR), a mid-training paradigm that leverages real-world GitHub pull requests as a training signal for repository-level editing. We introduce a scalable pipeline that converts noisy pull request diffs into Search/Replace edit blocks through reconstruction and validation, resulting in the largest publicly available corpus of 2 million pull requests spanning 12 programming languages. Using this training signal, we perform a mid-training stage followed by an agentless-aligned supervised fine-tuning process with error-driven data augmentation. On SWE-bench, our model significantly outperforms the instruction-tuned baseline, achieving absolute improvements of 13.6% on SWE-bench Lite and 12.3% on SWE-bench Verified. These results demonstrate that repository-level code understanding and editing capabilities can be effectively internalised into model weights under a simplified, agentless protocol, without relying on heavy inference-time scaffolding.

Modern AI coding assistants are becoming increasingly useful, but they still often struggle with real software projects, where fixing a problem may require understanding many files and making careful changes in the right places. In this paper, we study how to improve these abilities by learning from real developer activity on GitHub. We collect and clean a large number of pull requests, which are proposed code changes reviewed and accepted by developers, and turn them into reliable examples showing what problem was being solved and how the code was changed. We introduce Clean-PR, a dataset and training approach that helps an AI model learn how real software fixes are made. Instead of relying mainly on complex step-by-step agent systems at test time, our method teaches more of this knowledge directly into the model. Experiments show that models trained with Clean-PR are better at finding the right files, making valid code changes, and solving real software issues. This suggests that high-quality real-world developer data can make AI coding assistants more practical, reliable, and easier to use.