THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Talk
Learning to Learn without Gradient Descent by Gradient Descent
Yutian Chen · Matthew Hoffman · Sergio Gómez Colmenarejo · Misha Denil · Timothy Lillicrap · Matthew Botvinick · Nando de Freitas

Mon Aug 07 12:15 AM -- 12:33 AM (PDT) @ Darling Harbour Theatre
in Deep learning 4: learning to learn »

We learn recurrent neural network optimizers trained on simple synthetic functions by gradient descent. We show that these learned optimizers exhibit a remarkable degree of transfer in that they can be used to efficiently optimize a broad range of derivative-free black-box functions, including Gaussian process bandits, simple control objectives, global optimization benchmarks and hyper-parameter tuning tasks. Up to the training horizon, the learned optimizers learn to trade-off exploration and exploitation, and compare favourably with heavily engineered Bayesian optimization packages for hyper-parameter tuning.

[ PDF]  [ ss Summary/Notes]  [ Video] 

Author Information

Yutian Chen (DeepMind)
Matthew Hoffman (DeepMind)
Sergio Gómez Colmenarejo (Google DeepMind)
Misha Denil (University of Oxford)
Timothy Lillicrap (Google DeepMind)
Matthew Botvinick (DeepMind)
Nando de Freitas (DeepMind)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors