Timezone: »

 
Poster
GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models
Hanjing Wang · Man-Kit Sit · Congjie He · Ying Wen · Weinan Zhang · Jun Wang · Yaodong Yang · Luo Mai

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #216
in Poster Session 6 »

This paper introduces a distributed, GPU-centric experience replay system, GEAR, designed to perform scalable reinforcement learning (RL) with large sequence models (such as transformers). With such models, existing systems such as Reverb face considerable bottlenecks in memory, computation, and communication. GEAR, however, optimizes memory efficiency by enabling the memory resources on GPU servers (including host memory and device memory) to manage trajectory data. Furthermore, it facilitates decentralized GPU devices to expedite various trajectory selection strategies, circumventing computational bottlenecks. GEAR is equipped with GPU kernels capable of collecting trajectories using zero-copy access to host memory, along with remote-directed-memory access over InfiniBand, improving communication efficiency. Cluster experiments have shown that GEAR can achieve performance levels up to 6× greater than Reverb when training state-of-the-art large RL models. GEAR is open-sourced at https:// github.com/bigrl-team/gear.

[ PDF]  [ Poster] 

Author Information

Hanjing Wang (Shanghai Jiao Tong University)
Man-Kit Sit (The University of Edinburgh)
Congjie He (Informatics Forum, University of Edinburgh)
Ying Wen (Shanghai Jiao Tong University)
Weinan Zhang (Shanghai Jiao Tong University)
Jun Wang (University College London)
Yaodong Yang (Huawei UK)
Luo Mai (University of Edinburgh, University of Edinburgh)

More from the Same Authors