Timezone: »

Learning to Assemble with Large-Scale Structured Reinforcement Learning
Seyed Kamyar Seyed Ghasemipour · Satoshi Kataoka · Byron David · Daniel Freeman · Shixiang Gu · Igor Mordatch

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ Hall E #123

Assembly of multi-part physical structures is both a valuable end product for autonomous robotics, as well as a valuable diagnostic task for open-ended training of embodied intelligent agents. We introduce a naturalistic physics-based environment with a set of connectable magnet blocks inspired by children’s toy kits. The objective is to assemble blocks into a succession of target blueprints. Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter. Furthermore, assembly stresses agents' physical reasoning, bimanual coordination, and multi-step planning. We find that combination of large-scale reinforcement learning and graph-based policies is an effective recipe for agents that generalize to complex unseen blueprints in a zero-shot manner, and even operate in a reset-free setting without being trained to do so. Through extensive experiments, we highlight the importance of structured representations, contributions of multi-task vs. single-task learning, effects of curriculums, and discuss qualitative behaviors of trained agents.

Author Information

Seyed Kamyar Seyed Ghasemipour (University of Toronto)
Satoshi Kataoka (Google LLC)
Byron David (Google)
Daniel Freeman (Google Brain)
Shixiang Gu (Google)
Igor Mordatch (Google Brain)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors