Timezone: »

 
Learning Higher Order Skills that Efficiently Compose
Anthony Liu · Dong Ki Kim · Sungryull Sohn · Honglak Lee

@
in The Many Facets of Preference-Based Learning »
Event URL: https://openreview.net/forum?id=mEElj97a8C »

Hierarchical reinforcement learning allows an agent to effectively solve complex tasks by leveraging the compositional structures of tasks and executing a sequence of skills. However, our examination shows that prior work focuses on learning individual skills without considering how to efficiently combine them, which can lead to sub-optimal performance.To address this problem, we propose a novel framework, called second-order skills (SOS), for learning skills to facilitate the efficient execution of skills in sequence. Specifically, second order skills (which can be generalized to higher orders) aim to learn skills from an extended perspective that takes into account the next skill required to accomplish a task.We theoretically demonstrate that our method guarantees more efficient performance in the downstream task compared to previous approaches that do not consider second-order skills. Also, our empirical experiments show that learning second-order skills results in improved learning performance compared to state-of-the-art in baselines across diverse benchmark domains.

Author Information

Anthony Liu (University of Michigan)
Dong Ki Kim (MIT)

Dong-Ki Kim is a PhD candidate at Massachusetts Institute of Technology. His research focuses on multiagent reinforcement learning, developing algorithms that enable AI agents to interact with other simultaneously learning agents, share knowledge with other teammates, and learn robust policies against opponents. His work has received the best student paper honorable mention at AAAI'19 and featured in NVIDIA, WIRED, and MIT news. Previously, He completed a B.S. degree with Summa Cum Laude at Cornell University. He also worked at the Robotics Institute, Carnegie Mellon University, and Toyota Technological Institute at Chicago researching machine learning and robotics.

Sungryull Sohn (LG AI research center, Ann Arbor)
Honglak Lee (LG AI Research / U. Michigan)

More from the Same Authors