Timezone: »

 
Spotlight
Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning
Seyed Kamyar Seyed Ghasemipour · Satoshi Kataoka · Byron David · Daniel Freeman · Shixiang Gu · Igor Mordatch

Thu Jul 21 10:55 AM -- 11:00 AM (PDT) @ Hall G
in Applications »

Assembly of multi-part physical structures is both a valuable end product for autonomous robotics, as well as a valuable diagnostic task for open-ended training of embodied intelligent agents. We introduce a naturalistic physics-based environment with a set of connectable magnet blocks inspired by children’s toy kits. The objective is to assemble blocks into a succession of target blueprints. Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter. Furthermore, assembly stresses agents' multi-step planning, physical reasoning, and bimanual coordination. We find that the combination of large-scale reinforcement learning and graph-based policies-- surprisingly without any additional complexity -- is an effective recipe for training agents that not only generalize to complex unseen blueprints in a zero-shot manner, but even operate in a reset-free setting without being trained to do so. Through extensive experiments, we highlight the importance of large-scale training, structured representations, contributions of multi-task vs. single-task learning, as well as the effects of curriculums, and discuss qualitative behaviors of trained agents. Our accompanying project webpage can be found at: https://sites.google.com/view/learning-direct-assembly/home

Keywords: APP: Robotics · DL: Graph Neural Networks · RL: Deep RL · RL: Online · MISC: Transfer, Multitask and Meta-learning

Author Information

Seyed Kamyar Seyed Ghasemipour (University of Toronto)
Satoshi Kataoka (Google LLC)
Byron David (Google)
Daniel Freeman (Google Brain)
Shixiang Gu (Google)
Igor Mordatch (Google Brain)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors