Timezone: »

 
Poster
PolyGen: An Autoregressive Generative Model of 3D Meshes
Charlie Nash · Yaroslav Ganin · S. M. Ali Eslami · Peter Battaglia

Wed Jul 15 11:00 AM -- 11:45 AM & Thu Jul 16 12:00 AM -- 12:45 AM (PDT) @
in Poster Session 25 »

Polygon meshes are an efficient representation of 3D geometry, and are of central importance in computer graphics, robotics and games development. Existing learning-based approaches for object synthesis have avoided the challenges of working with 3D meshes, instead using alternative object representations that are more compatible with neural architectures and training approaches. We present PolyGen, a generative model of 3D objects which models the mesh directly, predicting vertices and faces sequentially using a Transformer-based architecture. Our model can condition on a range of inputs, including object classes, voxels, and images, and because the model is probabilistic it can produce samples that capture uncertainty in ambiguous scenarios. We show that the model is capable of producing high-quality, usable meshes, and establish log-likelihood benchmarks for the mesh-modelling task. We also evaluate the conditional models on surface reconstruction metrics against alternative methods, and demonstrate competitive performance despite not training directly on this task.

Keywords: Computer Vision · Deep Generative Models · Deep Sequence Models · Structured Prediction · Deep Learning - Generative Models and Autoencoders

Author Information

Charlie Nash (DeepMind)
Yaroslav Ganin (DeepMind)
S. M. Ali Eslami (DeepMind)
S. M. Ali Eslami

S. M. Ali Eslami is a staff research scientist at DeepMind working on problems related to artificial intelligence. Prior to that, he was a post-doctoral researcher at Microsoft Research in Cambridge. He did his PhD in the School of Informatics at the University of Edinburgh, during which he was also a visiting researcher in the Visual Geometry Group at the University of Oxford. His research is focused on figuring out how we can get computers to learn with less human supervision.

Peter Battaglia (DeepMind)

More from the Same Authors