Timezone: »
Inspired by recent successes of Monte-Carlo tree search (MCTS) in a number of artificial intelligence (AI) application domains, we propose a reinforcement learning (RL) technique that iteratively applies MCTS on batches of small, finite-horizon versions of the original infinite-horizon Markov decision process. The terminal condition of the finite-horizon problems, or the leaf-node evaluator of the decision tree generated by MCTS, is specified using a combination of an estimated value function and an estimated policy function. The recommendations generated by the MCTS procedure are then provided as feedback in order to refine, through classification and regression, the leaf-node evaluator for the next iteration. We provide the first sample complexity bounds for a tree search-based RL algorithm. In addition, we show that a deep neural network implementation of the technique can create a competitive AI agent for the popular multi-player online battle arena (MOBA) game King of Glory.
Author Information
Daniel Jiang (Facebook)
Emmanuel Ekwedike (Princeton University)
Han Liu (Northwestern)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Poster: Feedback-Based Tree Search for Reinforcement Learning »
Wed. Jul 11th 04:15 -- 07:00 PM Room Hall B #162
More from the Same Authors
-
2023 Poster: Feature Programming for Multivariate Time Series Prediction »
Alex Reneau · Jerry Yao-Chieh Hu · Ammar Gilani · Han Liu -
2022 Poster: Bregman Proximal Langevin Monte Carlo via Bregman--Moreau Envelopes »
Tim Tsz-Kit Lau · Han Liu -
2022 Spotlight: Bregman Proximal Langevin Monte Carlo via Bregman--Moreau Envelopes »
Tim Tsz-Kit Lau · Han Liu -
2020 Poster: Lookahead-Bounded Q-learning »
Ibrahim El Shar · Daniel Jiang -
2019 Poster: Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI »
Lei Han · Peng Sun · Yali Du · Jiechao Xiong · Qing Wang · Xinghai Sun · Han Liu · Tong Zhang -
2019 Oral: Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI »
Lei Han · Peng Sun · Yali Du · Jiechao Xiong · Qing Wang · Xinghai Sun · Han Liu · Tong Zhang -
2018 Poster: Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents »
Kaiqing Zhang · Zhuoran Yang · Han Liu · Tong Zhang · Tamer Basar -
2018 Oral: Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents »
Kaiqing Zhang · Zhuoran Yang · Han Liu · Tong Zhang · Tamer Basar