THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Talk
RobustFill: Neural Program Learning under Noisy I/O
Jacob Devlin · Jonathan Uesato · Surya Bhupatiraju · Rishabh Singh · Abdelrahman Mohammad · Pushmeet Kohli

Mon Aug 07 10:48 PM -- 11:06 PM (PDT) @ C4.1
in ML and programming »

The problem of automatically generating a computer program from some specification has been studied since the early days of AI. Recently, two competing approaches for 'automatic program learning' have received significant attention: (1) 'neural program synthesis', where a neural network is conditioned on input/output (I/O) examples and learns to generate a program, and (2) 'neural program induction', where a neural network generates new outputs directly using a latent program representation. Here, for the first time, we directly compare both approaches on a large-scale, real-world learning task and we additionally contrast to rule-based program synthesis, which uses hand-crafted semantics to guide the program generation. Our neural models use a modified attention RNN to allow encoding of variable-sized sets of I/O pairs, which achieve 92% accuracy on a real-world test set, compared to the 34% accuracy of the previous best neural synthesis approach. The synthesis model also outperforms a comparable induction model on this task, but we more importantly demonstrate that the strength of each approach is highly dependent on the evaluation metric and end-user application. Finally, we show that we can train our neural models to remain very robust to the type of noise expected in real-world data (e.g., typos), while a highly-engineered rule-based system fails entirely.

[ PDF]  [ ss Summary/Notes]  [ Video] 

Author Information

Jacob Devlin (Microsoft Research)
Jonathan Uesato (MIT)
Surya Bhupatiraju (MIT)
Rishabh Singh (Microsoft Research)
Abdelrahman Mohammad (Microsoft)
Pushmeet Kohli (Microsoft Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors