Significant strides have been made toward designing better generative models in recent years. Despite this progress, however, state-of-the-art approaches are still largely unable to capture complex global structure in data. For example, images of buildings typically contain spatial patterns such as windows repeating at regular intervals; state-of-the-art generative methods can't easily reproduce these structures. We propose to address this problem by incorporating programs representing global structure into the generative model---e.g., a 2D for-loop may represent a configuration of windows. Furthermore, we propose a framework for learning these models by leveraging program synthesis to generate training data. On both synthetic and real-world data, we demonstrate that our approach is substantially better than the state-of-the-art at both generating and completing images that contain global structure.
Halley R Young (University of Pennsylvania)
I am a second-year PhD student at University of Pennsylvania. I work in the programming languages group under the guidance of Mayur Naik. My research interests include the intersection of deep learning with programming languages, generative models of visual and musical programs, and computer science and the arts.
Osbert Bastani (University of Pennsylvania)
Mayur Naik (University of Pennsylvania)
Related Events (a corresponding poster, oral, or spotlight)
2019 Poster: Learning Neurosymbolic Generative Models via Program Synthesis »
Thu Jun 13th 01:30 -- 04:00 AM Room Pacific Ballroom
More from the Same Authors
2020 Poster: Robust Black Box Explanations Under Distribution Shift »
Himabindu Lakkaraju · Nino Arsov · Osbert Bastani
2020 Poster: Generating Programmatic Referring Expressions via Program Synthesis »
Jiani Huang · Calvin Smith · Osbert Bastani · Rishabh Singh · Aws Albarghouthi · Mayur Naik