Fast adaptation: a CL perspective

In this talk I will focus on the theme of the workshop – fast adaptation – and discuss a few challenges and perspectives, looking at the problem from a Continual Learning (CL) and Learning Dynamics point of view. I will start by framing the concept of fast adaptation in the context of CL and specifically talk about the role of catastrophic forgetting (CF) . While catastrophic forgetting seems to be at odds with adaptation, I will argue that there is a sense in which solving catastrophic forgetting should be necessary to improve learning efficiency. Specifically I will focus on interference as a mechanism that slows down learning for modern deep learning. In the last part of my talk I will discuss compositionality as an alternative way of thinking of adaptation, arguing that modern models struggle with compositional learning and how compositionality could go beyond solving interference.