Invited Talk
Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward

Chinchillas, Flamingos, and Gatos: Few-Shot Learning through Pre-training

Oriol Vinyals


Three of our recent sequence models - Chinchilla, Flamingo, and Gato - leveraged one another and combined several state-of-the-art pre-training techniques. This talk will describe how this combination yielded even stronger capabilities to achieving complex tasks, in the few-shot setting, beyond what could have been expected from their training regimes.

Chat is not available.