Timezone: »

 
Undo Maps: A Tool for Adapting Policies to Perceptual Distortions
Abhi Gupta · Ted Moskovitz · David Alvarez-Melis · Aldo Pacchiano

@
in New Frontiers in Learning, Control, and Dynamical Systems »
Event URL: https://openreview.net/forum?id=0Rk3N5D8cS »

People adapt to changes in their visual field all the time, like when their vision is occluded while driving. Agents trained with RL struggle to do the same. Here, we address how to transfer knowledge acquired in one domain to another when the domains differ in their state representation. For example, a policy may have been trained in an environment where states were represented as colored images, but we would now like to deploy this agent in a domain where images appear black-and-white. We propose \textsc{Tail}--task-agnostic imitation learning--a framework which learns to undo these kinds of changes between domains in order to achieve transfer. This enables an agent, regardless of the task it was trained for, to adapt to perceptual distortions by first mapping the states in the new domain, such as gray-scale images, back to the original domain where they appear in color, and then by acting with the same policy. Our procedure depends on an optimal transport formulation between trajectories in the two domains, shows promise in simple experimental settings, and resembles algorithms from imitation learning.

Author Information

Abhi Gupta (Massachusetts Institute of Technology)
Ted Moskovitz (Gatsby Computational Neuroscience Unit)
David Alvarez-Melis (MSR / Harvard)
David Alvarez-Melis

I am currently a senior researcher at Microsoft Research New England. Starting in 2023, I will be joining Harvard University as an assistant professor of Computer Science and Applied Mathematics.

Aldo Pacchiano (Broad Institute)

More from the Same Authors