Timezone: »
Language is one of the most impressive human accomplishments and is believed to be the core to our ability to learn, teach, reason and interact with others. Yet, current state-of-the-art reinforcement learning agents are unable to use or understand human language at all. The ability to integrate and learn from language, in addition to rewards and demonstrations, has the potential to improve the generalization, scope and sample efficiency of agents. Furthermore, many real-world tasks, including personal assistants and general household robots, require agents to process language by design, whether to enable interaction with humans, or simply use existing interfaces. The aim of our workshop is to advance this emerging field of research by bringing together researchers from several diverse communities to discuss recent developments in relevant research areas such as instruction following and embodied language learning, and identify the most important challenges and promising research avenues.
Sat 7:00 a.m. - 7:10 a.m.
|
Welcome Remarks
|
🔗 |
Sat 7:10 a.m. - 7:40 a.m.
|
Invited Talk: Angeliki Lazaridou
(
Talk
)
link »
SlidesLive Video » The ability to cooperate through language is a defining feature of humans. As the perceptual, motory and planning capabilities of deep artificial networks increase, researchers are studying whether they also can develop a shared language to interact. In this talk, I will highlight recent advances in this field but also common headaches (or perhaps limitations) with respect to experimental setup and evaluation of emergent communication. Towards making multi-agent communication a building block of human-centric AI, and by drawing from my own recent work, I will discuss approaches on making emergent communication relevant for human-agent communication in natural language. |
Angeliki Lazaridou 🔗 |
Sat 7:40 a.m. - 8:10 a.m.
|
Invited Talk: Arthur Szlam
(
Talk
)
link »
I will discuss our progress on a research program aimed at building a Minecraft assistant. I will cover the tools and platform we have built allowing players to interact with the agents and to record those interactions, and the data we have collected. I will also cover the design of our current agent, from which we (and hopefully others) can iterate. |
Nantas Nardelli 🔗 |
Sat 8:10 a.m. - 8:30 a.m.
|
Coffee break 1 link » | 🔗 |
Sat 8:30 a.m. - 9:15 a.m.
|
Poster session 1
(
Poster Session
)
link »
Check out the papers and their short presentations here: https://larel-ws.github.io/accepted-papers/ Meet the authors in LaReL's Gather Town: https://tinyurl.com/gather-larel |
Nantas Nardelli 🔗 |
Sat 9:15 a.m. - 10:15 a.m.
|
Lunch Break link » | 🔗 |
Sat 10:15 a.m. - 10:45 a.m.
|
Invited Talk: Felix Hill
(
Talk
)
link »
SlidesLive Video » Models like BERT or GPT-2 can do amazing things with language, and this raises the interesting question of whether such text-based models could ever really "understand" it. One clear difference between BERT-understanding and human understanding is that BERT doesn't learn to connect language to its actions or its perception of the world it inhabits. I'll discuss an alternative approach to language understanding in which a neural-network-based agent is trained to associate words and phrases with things that it learns to see and do. First, I'll provide some evidence for the promise of this approach by showing that the interactive, first-person perspective of an agent affords it with a particular inductive bias that helps it to extend its training experience to generalize to out-of-distribution settings in ways that seem natural or 'systematic'. Second, I'll show the amount of 'propositional' (i.e. linguistic) knowledge that emerges in the internal states of the agent as it interacts with the world can be increased significantly by it learning to make predictions about observations multiple timesteps into the future. This underlines some important common ground between the agent-based and BERT-style approaches: both attest to the power of prediction and the importance of context in acquiring semantic representations. Finally, I'll connect BERT and agent-based learning in a more literal way, by showing how an agent endowed with BERT representations can achieve substantial (zero-shot) transfer from template-based language to noisy natural instructions given by humans with access to the agent's world. |
Feilx Hill 🔗 |
Sat 10:45 a.m. - 11:15 a.m.
|
Invited Talk: Karthik Narasimhan
(
Talk
)
link »
SlidesLive Video » In recent years, reinforcement learning (RL) has been used with considerable success in games and robotics as well as language understanding applications like dialog systems. However, the question of what language can provide for RL remains relatively under-explored. In this talk, I make the case that leveraging language will be essential to developing general-purpose interactive agents that can perform more than a single task and operate in scenarios beyond the ones they are trained on. Natural language allows us to incorporate more semantic structure into the RL framework while also making it easier to obtain guidance from humans. Specifically, I will show how several parts of the traditional RL setup (e.g. transitions, rewards, actions, goals) can be expressed in language to build agents that can handle combinatorially large spaces as well as generalize to unseen subspaces in each of these aspects. |
Karthik Narasimhan 🔗 |
Sat 11:15 a.m. - 11:45 a.m.
|
Invited Talk: Yoav Artzi
(
Talk
)
link »
SlidesLive Video » I will discuss the task of executing natural language instructions with a physical robotic agent. In contrast to existing work, we do not engineer formal representations of language meaning or the robot environment. Instead, we learn to directly map raw observations and language to low-level continuous control of a quadcopter drone. We use an interpretable neural network model that mixes learned representations with differentiable geometric operations. For training, we introduce Supervised and Reinforcement Asynchronous Learning (SuReAL), a learning algorithm that utilizes supervised and reinforcement learning processes that constantly interact to learn robust reasoning with limited data. Our learning algorithm uses demonstrations and a plan-following intrinsic reward signal. While we do not require any real-world autonomous flight during learning, our model works effectively both in simulation and the real environment. |
Yoav Artzi 🔗 |
Sat 11:45 a.m. - 12:05 p.m.
|
Coffee break 2 link » | 🔗 |
Sat 12:05 p.m. - 12:50 p.m.
|
Poster session 2
(
Poster Session
)
link »
Check out the papers and their short presentations here: https://larel-ws.github.io/accepted-papers/ Meet the authors in LaReL's Gather Town: https://tinyurl.com/gather-larel |
Nantas Nardelli 🔗 |
Sat 12:50 p.m. - 1:00 p.m.
|
Short Break link » | 🔗 |
Sat 1:00 p.m. - 1:30 p.m.
|
Invited Talk: Marc-Alexandre Côté
(
Talk
)
link »
SlidesLive Video » Text-based games are complex, interactive simulations in which text describes the game state and players make progress by entering text commands. They are fertile ground for language-focused machine learning research. In addition to language understanding, successful play requires skills like long-term memory and planning, exploration (trial and error), and common sense. The talk will introduce TextWorld, a sandbox learning environment for the training and evaluation of RL agents on text-based games. Its generative mechanisms give precise control over the difficulty, scope, and language of constructed games, and can be used to study generalization and transfer learning. This talk will also give an overview of the recent attempts to solve text-based games either using reinforcement learning or more handcrafted approaches. |
Marc-Alexandre Côté 🔗 |
Sat 1:30 p.m. - 2:00 p.m.
|
Invited Talk: Alison Gopnik
(
Talk
)
link »
SlidesLive Video » Understanding, learning and reasoning with abstract relations, like same and different or bigger and smaller, is challenging. We show that in an RL like causal learning task, very young children, 18-30 month olds, can learn both same and different relations and the functions becoming bigger and becoming smaller, generalize those relations to brand new and perceptually different objects, and use them to solve novel tasks. We suggest that both abstract causal representations, similar to causal graphical models, and early language may support this knowledge and learning. |
Nantas Nardelli 🔗 |
Sat 2:00 p.m. - 2:10 p.m.
|
Closing Remarks
|
🔗 |
Author Information
Nantas Nardelli (University of Oxford)
Jelena Luketina (The University of Oxford)
Nantas Nardelli (University of Oxford)
Jakob Foerster (Facebook)
Victor Zhong (University of Washington)
Jacob Andreas (MIT)
Tim Rocktäschel (Facebook, UCL)
Edward Grefenstette (Facebook, UCL)
Tim Rocktäschel (Facebook AI Research & University College London)
More from the Same Authors
-
2023 : Do LLMs selectively encode the goal of an agent's reach? »
Laura Ruis · Arduin Findeis · Herbie Bradley · Hossein A. Rahmani · Kyoung Whan Choe · Edward Grefenstette · Tim Rocktäschel -
2023 Poster: Guiding Pretraining in Reinforcement Learning with Large Language Models »
Yuqing Du · Olivia Watkins · Zihan Wang · Cédric Colas · Trevor Darrell · Pieter Abbeel · Abhishek Gupta · Jacob Andreas -
2023 Oral: Human-Timescale Adaptation in an Open-Ended Task Space »
Jakob Bauer · Kate Baumli · Feryal Behbahani · Avishkar Bhoopchand · Natalie Bradley-Schmieg · Michael Chang · Natalie Clay · Adrian Collister · Vibhavari Dasagi · Lucy Gonzalez · Karol Gregor · Edward Hughes · Sheleem Kashem · Maria Loks-Thompson · Hannah Openshaw · Jack Parker-Holder · Shreya Pathak · Nicolas Perez-Nieves · Nemanja Rakicevic · Tim Rocktäschel · Yannick Schroecker · Satinder Singh · Jakub Sygnowski · Karl Tuyls · Sarah York · Alexander Zacherl · Lei Zhang -
2023 Poster: PromptBoosting: Black-Box Text Classification with Ten Forward Passes »
Bairu Hou · Joe O'Connor · Jacob Andreas · Shiyu Chang · Yang Zhang -
2023 Poster: Human-Timescale Adaptation in an Open-Ended Task Space »
Jakob Bauer · Kate Baumli · Feryal Behbahani · Avishkar Bhoopchand · Natalie Bradley-Schmieg · Michael Chang · Natalie Clay · Adrian Collister · Vibhavari Dasagi · Lucy Gonzalez · Karol Gregor · Edward Hughes · Sheleem Kashem · Maria Loks-Thompson · Hannah Openshaw · Jack Parker-Holder · Shreya Pathak · Nicolas Perez-Nieves · Nemanja Rakicevic · Tim Rocktäschel · Yannick Schroecker · Satinder Singh · Jakub Sygnowski · Karl Tuyls · Sarah York · Alexander Zacherl · Lei Zhang -
2022 Poster: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2022 Spotlight: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2022 Poster: Modeling Strong and Human-Like Gameplay with KL-Regularized Search »
Athul Paul Jacob · David Wu · Gabriele Farina · Adam Lerer · Hengyuan Hu · Anton Bakhtin · Jacob Andreas · Noam Brown -
2022 Spotlight: Modeling Strong and Human-Like Gameplay with KL-Regularized Search »
Athul Paul Jacob · David Wu · Gabriele Farina · Adam Lerer · Hengyuan Hu · Anton Bakhtin · Jacob Andreas · Noam Brown -
2021 Poster: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2021 Spotlight: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2020 : Invited Talk: Alison Gopnik »
Nantas Nardelli -
2020 : Poster session 2 »
Nantas Nardelli -
2020 : Poster session 1 »
Nantas Nardelli -
2020 : Invited Talk: Arthur Szlam »
Nantas Nardelli -
2020 : The NetHack Learning Environment Q&A »
Tim Rocktäschel · Katja Hofmann -
2020 : The NetHack Learning Environment »
Tim Rocktäschel -
2020 Poster: Learning Reasoning Strategies in End-to-End Differentiable Proving »
Pasquale Minervini · Sebastian Riedel · Pontus Stenetorp · Edward Grefenstette · Tim Rocktäschel -
2019 Poster: A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs »
Jingkai Mao · Jakob Foerster · Tim Rocktäschel · Maruan Al-Shedivat · Gregory Farquhar · Shimon Whiteson -
2019 Oral: A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs »
Jingkai Mao · Jakob Foerster · Tim Rocktäschel · Maruan Al-Shedivat · Gregory Farquhar · Shimon Whiteson -
2018 Poster: Progress & Compress: A scalable framework for continual learning »
Jonathan Richard Schwarz · Wojciech Czarnecki · Jelena Luketina · Agnieszka Grabska-Barwinska · Yee Teh · Razvan Pascanu · Raia Hadsell -
2018 Oral: Progress & Compress: A scalable framework for continual learning »
Jonathan Richard Schwarz · Wojciech Czarnecki · Jelena Luketina · Agnieszka Grabska-Barwinska · Yee Teh · Razvan Pascanu · Raia Hadsell -
2017 Poster: Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning »
Jakob Foerster · Nantas Nardelli · Gregory Farquhar · Triantafyllos Afouras · Phil Torr · Pushmeet Kohli · Shimon Whiteson -
2017 Talk: Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning »
Jakob Foerster · Nantas Nardelli · Gregory Farquhar · Triantafyllos Afouras · Phil Torr · Pushmeet Kohli · Shimon Whiteson -
2017 Poster: Programming with a Differentiable Forth Interpreter »
Matko Bošnjak · Tim Rocktäschel · Jason Naradowsky · Sebastian Riedel -
2017 Talk: Programming with a Differentiable Forth Interpreter »
Matko Bošnjak · Tim Rocktäschel · Jason Naradowsky · Sebastian Riedel