Timezone: »

 
Spotlight
EqR: Equivariant Representations for Data-Efficient Reinforcement Learning
Arnab Kumar Mondal · Vineet Jain · Kaleem Siddiqi · Siamak Ravanbakhsh

Thu Jul 21 12:55 PM -- 01:00 PM (PDT) @ Room 309
in Reinforcement Learning/Optimization »

We study a variety of notions of equivariance as an inductive bias in Reinforcement Learning (RL). In particular, we propose new mechanisms for learning representations that are equivariant to both the agent’s action, as well as symmetry transformations of the state-action pairs. Whereas prior work on exploiting symmetries in deep RL can only incorporate predefined linear transformations, our approach allows non-linear symmetry transformations of state-action pairs to be learned from the data. This is achieved through 1) equivariant Lie algebraic parameterization of state and action encodings, 2) equivariant latent transition models, and 3) the incorporation of symmetry-based losses. We demonstrate the advantages of our method, which we call Equivariant representations for RL (EqR), for Atari games in a data-efficient setting limited to 100K steps of interactions with the environment.

Keywords: DL: Self-Supervised Learning · RL: Deep RL · RL: Function Approximation · MISC: Representation Learning

Author Information

Arnab Kumar Mondal (Mcgill University)
Arnab Kumar Mondal

I am a third-year Ph.D. candidate in Computer Science at Mcgill University and Mila - Quebec Artificial Intelligence Institute, jointly supervised by Prof. Siamak Ravanbakhsh and Prof. Kaleem Siddiqi. My primary areas of interest include equivariant representation learning, self-supervised learning and reinforcement learning. Before moving to Montreal, I did my undergraduate studies in Electronics and Electrical Communication Engineering from the Indian Institute of Technology, Kharagpur. I have worked on VLSI engineering, Robotics, Computer Vision, Embedded systems, and Free-form Lens design during my undergrad. When I am not working, I like to spend time in nature, stargaze with my close friends and talk about life. I love mountains and lakes and have been on a few Himalayan treks. Next on my list is Banff National Park.

Vineet Jain (McGill University, Mila)
Kaleem Siddiqi (McGill University)
Siamak Ravanbakhsh (McGill - Mila)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors