Invited Talk
in
Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward

Chinchillas, Flamingos, and Gatos: Few-Shot Learning through Pre-training

Oriol Vinyals

2022 Invited Talk
in
Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward

Abstract

Three of our recent sequence models - Chinchilla, Flamingo, and Gato - leveraged one another and combined several state-of-the-art pre-training techniques. This talk will describe how this combination yielded even stronger capabilities to achieving complex tasks, in the few-shot setting, beyond what could have been expected from their training regimes.

Speaker

Oriol Vinyals

Oriol Vinyals is a Research Scientist at Google. He works in deep learning with the Google Brain team. Oriol holds a Ph.D. in EECS from University of California, Berkeley, and a Masters degree from University of California, San Diego. He is a recipient of the 2011 Microsoft Research PhD Fellowship. He was an early adopter of the new deep learning wave at Berkeley, and in his thesis he focused on non-convex optimization and recurrent neural networks. At Google Brain he continues working on his areas of interest, which include artificial intelligence, with particular emphasis on machine learning, language, and vision.

Video

Chat is not available.