Invited Talk: Razvan Pascanu "Continual Learning from an Optimization/Learning-dynamics perspective"

Continual learning is usually described through a list of desiderata, however some of the "wants" on this list are in contradiction with each other, hence a solution to continual learning implies finding suitable trade-offs between the different objectives. Such trade-offs can be given by grounding ourselves into a particular domain or set of tasks. Alternatively, I believe, one can also rely on framing or looking at continual learning through different perspectives to gain this grounding. In this talk I'm looking at optimization and learning dynamics. From this perspective, continual learning can be seen as looking for a more suitable credit assignment mechanism for learning, one that does not rely on tug-of-war dynamics that result from gradient based optimization techniques. I exemplify in what sense this grounds us, and present a few recent projects I've been involved in that could be thought of as looking at continual learning from this perspective.