Skip to yearly menu bar Skip to main content


CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms

Shengyi Huang · Rousslan Fernand Julien Dossa · Chang Ye · Jeff Braga · Dipam Chakraborty · Kinal Mehta · João Madeira Araujo

Exhibit Hall 1 #113


CleanRL is an open-source library that provides high-quality single-file implementations of Deep Reinforcement Learning (DRL) algorithms. These single-file implementations are self-contained algorithm variant files such as,, and that individually include all algorithm variant's implementation details. Such a paradigm significantly reduces the complexity and the lines of code (LOC) in each implemented variant, which makes them quicker and easier to understand. This paradigm gives the researchers the most fine-grained control over all aspects of the algorithm in a single file, allowing them to prototype novel features quickly. Despite having succinct implementations, CleanRL's codebase is thoroughly documented and benchmarked to ensure performance is on par with reputable sources. As a result, CleanRL produces a repository tailor-fit for two purposes: 1) understanding all implementation details of DRL algorithms and 2) quickly prototyping novel features. CleanRL's source code can be found at

Chat is not available.