OffWorld Gym: Open-Access Physical Robotics Environment for Real-World Reinforcement Learning Benchmark and Research

Success stories of applied machine learning can be traced back to the datasets and environments that were put forward as challenges for the community. The challenge that the community sets as a benchmark is usually the challenge that the community eventually solves. The ultimate challenge of reinforcement learning research is to train real agents to operate in the real environment, but there is no common real-world benchmark to track the progress of RL on physical robotic systems. To address this issue we have created a physical RL benchmark -- a collection of real-world environments for reinforcement learning in robotics with free public remote access. In this work, we introduce four tasks in two environments and the experimental results on one of them that demonstrate the feasibility of learning on a real robotic system. We train a mobile robot end-to-end to solve a visual navigation task relying solely on camera input and without the access to location information. Close integration into existing ecosystem allows the community to start using the proposed system without any prior experience in robotics and takes away the burden of managing a physical robotics system, abstracting it under a familiar API.