Toward Training Superintelligent Software Agents through Self-Play SWE-RL

While current software agents powered by large language models (LLMs) and reinforcement learning (RL) can boost programmer productivity, their reliance on human-curated training data and environments creates a fundamental barrier to superintelligence. In this paper, we present Self-play SWE-RL (SSR), a first step toward training superintelligent software agents under minimal data assumptions. SSR requires only access to sandboxed repositories with source code and dependencies, no need for human-labeled issues or test commands. Grounded in real-world codebases, a single LLM agent is trained via RL in a self-play setting to inject and repair increasingly complex bugs. The bugs are formally specified by test suite improvements proposed by the agent rather than natural language issue descriptions. On the SWE-bench Verified and SWE-Bench Pro benchmarks, SSR achieves clear self-improvement (+10.4 and +7.8 points) and consistently outperforms the human-data baseline throughout training, generalizing to natural language bug descriptions not seen in training. Overall, our results point toward a paradigm where agents autonomously gather extensive learning experiences from real software repositories, ultimately enabling superintelligent systems that exceed human capabilities in understanding, modifying, and creating software from scratch.

Software engineering agents powered by large language models can already help programmers write and debug code. However, most current systems still depend heavily on large collections of human-written bug reports, tests, and training examples. This creates a limit on how much these systems can improve using only human-provided data. In this work, we explore whether software agents can improve by learning from their own experience instead. We introduce Self-play SWE-RL (SSR), a training framework where an AI agent learns by both creating bugs in real software repositories and then attempting to fix them. The system only requires access to source code and runnable environments, not human-written issue descriptions or manually prepared tests. During training, the agent learns to generate increasingly realistic and challenging software problems while also improving its ability to solve them. We evaluate SSR on widely used software engineering benchmarks and find that it steadily improves over training and outperforms a comparable system trained using standard human-curated data. Our results suggest that self-play may become a scalable way to train future software engineering agents by allowing them to autonomously generate large amounts of learning experience from real-world codebases.