Chinchillas, Flamingos, and Gatos: Few-Shot Learning through Pre-training
Oriol Vinyals
2022 Invited Talk
in
Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward
in
Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward
Abstract
Three of our recent sequence models - Chinchilla, Flamingo, and Gato - leveraged one another and combined several state-of-the-art pre-training techniques. This talk will describe how this combination yielded even stronger capabilities to achieving complex tasks, in the few-shot setting, beyond what could have been expected from their training regimes.
Video
Chat is not available.
Successful Page Load