Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both these things. In this talk, I present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based rogue like game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open-source at https://github.com/facebookresearch/nle .

Please join using the Zoom link and post your questions on Rocketchat.

The field of reinforcement learning is pushed forwards by the presence of challenging environments. Over the years, the complexity of these environments has continued to increase, but the question is how can we continue to push the complexity of environments with respect to the optimal policy complexity in a scalable manner. Here I will discuss using multi-agent environments to create more open-ended environments, and discuss examples of our work to move in this direction with Capture the Flag and Starcraft 2. Finally I will discuss some future directions for generating even more open-ended environments to further push our RL algorithms.

Please join using the Zoom link and post your questions on Rocketchat.

I will present work done by my group on defining a collaborative construction task that allows us to use the Minecraft platform to study situated natural language generation and understanding. In this task, one player (the Architect) needs to instruct another (the Builder) via a chat interface to construct a given target structure that only the Architect is shown. I will discuss what makes this task interesting and challenging. I will also describe models that we have developed for the Architect and the Builder role, and discuss what remains to be done to create agents that can solve this task.

Please join using the Zoom link and post your questions on Rocketchat.

I will focus on the problem of executing natural language instructions in a collaborative environment. I will propose the task of learning to follow sequences of instructions in a collaborative scenario, where two agents, a leader and a follower, execute actions in the environment and the leader controls the follower using natural language. To study this problem, we build CerealBar, a multi-player 3D game where a leader instructs a follower, and both act in the environment together to accomplish complex goals. I will focus on learning an autonomous follower that executes the instructions of a human leader. I will briefly describe a model to address this problem, and a learning method that relies on static recorded human-human interactions, while still learning to recover from cascading errors between instructions.

Please join using the Zoom link and post your questions on Rocketchat.

Agents tackling complex problems in open environments often benefit from the ability to construct knowledge. Learning to independently solve sub-tasks and form models of the world can help agents progress in solving challenging problems. In this talk, we draw attention to challenges that arise when evaluating an agent’s knowledge, specifically focusing on methods that express an agent’s knowledge as predictions. Using the General Value Function framework we highlight the distinction between useful knowledge and strict measures of accuracy. Having identified challenges in assessing an agent’s knowledge, we propose a possible evaluation approach that is compatible with large and open worlds.

Please join using the Zoom link and post your questions on Rocketchat.

The research community is gradually coming to a realization that policies trained arcade-like video games are very limited. They overfit badly and are not going to take us far along the way to some sort of general intelligence. This should perhaps not be surprising, given that such games generally have tightly defined tasks, fixed perspectives, and generally static worlds. More and more attention is therefore given to games that are in some sense open-ended or feature open worlds. Could such games be the solution to our problems, allowing the development of more general artificial intelligence? Perhaps, but basing competitions or benchmarks on open-ended games is not going to be easy, as the very features which make for a good benchmark are the same that lead to brittle policies. Shoe-horning open-world games into a standard RL framework is unlikely to be the best option for going forward. Many of the most interesting opportunities for developing intelligent behavior is likely to come from agents constructing their own challenges and environments. The boundary between playing a game and constructing a world is not well-defined: I will give examples from where the same RL setup was used to play SimCity and to develop game levels. I will also briefly introduce the Generative Design in Minecraft Competition, which focuses on building believable settlements.

Please join using the Zoom link and post your questions on Rocketchat.

Hey all! Our poster session is hosted in a virtual town. To go to the poster session, please go to : [ protected link dropped ]

and use the password: flatfish

Join us for a panel discussion with our invited speakers