Skip to yearly menu bar Skip to main content


Poster

Learning to Compile Programs to Neural Networks

Logan Weber · Jesse Michel · Alex Renda · Michael Carbin


Abstract: A *neural surrogate* is a neural network that mimics the behavior of a program. Neural surrogates of programs have been used to automatically tune program inputs, adapt programs to new settings, and accelerate computations. Neural surrogates have traditionally been developed by training on input-output examples for a single program. Language models present another approach wherein a model is trained on a single, large dataset then directly consumes program text, to act as a neural surrogate of the program. Having the language model as both the neural surrogate generator and the neural surrogate, however, poses a tradeoff of limited accuracy or excessive resource consumption. We present *neural surrogate compilation*, a technique for producing neural surrogates directly from program text without coupling neural surrogate generation and execution. We implement neural surrogate compilers using hypernetworks trained on a dataset of C programs and find they produce neural surrogates that are $2.56$-$5.51\times$ as data-efficient and train in $1.52$-$3.34\times$ fewer epochs than neural surrogates trained from scratch.

Live content is unavailable. Log in and register to view live content