Timezone: »
Generative autoencoders offer a promising approach for controllable text generation by leveraging their learned sentence representations. However, current models struggle to maintain coherent latent spaces required to perform meaningful text manipulations via latent vector operations. Specifically, we demonstrate by example that neural encoders do not necessarily map similar sentences to nearby latent vectors. A theoretical explanation for this phenomenon establishes that high-capacity autoencoders can learn an arbitrary mapping between sequences and associated latent representations. To remedy this issue, we augment adversarial autoencoders with a denoising objective where original sentences are reconstructed from perturbed versions (referred to as DAAE). We prove that this simple modification guides the latent space geometry of the resulting model by encouraging the encoder to map similar texts to similar latent representations. In empirical comparisons with various types of autoencoders, our model provides the best trade-off between generation quality and reconstruction capacity. Moreover, the improved geometry of the DAAE latent space enables \textit{zero-shot} text style transfer via simple latent vector arithmetic.
Author Information
Tianxiao Shen (MIT)
Jonas Mueller (Amazon Web Services)
Regina Barzilay (MIT CSAIL)

Regina Barzilay is an Israeli-American computer scientist. She is a professor at the Massachusetts Institute of Technology and a faculty lead for artificial intelligence at the MIT Jameel Clinic. Her research interests are in natural language processing and applications of deep learning to chemistry and oncology.
Tommi Jaakkola (MIT)
More from the Same Authors
-
2021 : Multimodal AutoML on Structured Tables with Text Fields »
Xingjian Shi · Jonas Mueller · Nick Erickson · Mu Li · Alex Smola -
2021 : Continuous Doubly Constrained Batch Reinforcement Learning »
Rasool Fakoor · Jonas Mueller · Kavosh Asadi · Pratik Chaudhari · Alex Smola -
2022 : Adaptive Interest for Emphatic Reinforcement Learning »
Martin Klissarov · Rasool Fakoor · Jonas Mueller · Kavosh Asadi · Taesup Kim · Alex Smola -
2022 : Back to the Basics: Revisiting Out-of-Distribution Detection Baselines »
Johnson Kuan · Jonas Mueller -
2023 Poster: PFGM++: Unlocking the Potential of Physics-Inspired Generative Models »
Yilun Xu · Ziming Liu · Yonglong Tian · Shangyuan Tong · Max Tegmark · Tommi Jaakkola -
2023 Poster: Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models »
Guanhua Zhang · Jiabao Ji · Yang Zhang · Mo Yu · Tommi Jaakkola · Shiyu Chang -
2023 Poster: SE(3) diffusion model with application to protein backbone generation »
Jason Yim · Brian Trippe · Valentin De Bortoli · Emile Mathieu · Arnaud Doucet · Regina Barzilay · Tommi Jaakkola -
2022 : Model-Agnostic Label Quality Scoring to Detect Real-World Label Errors »
Jonas Mueller -
2022 Poster: Learning Stable Classifiers by Transferring Unstable Features »
Yujia Bao · Shiyu Chang · Regina Barzilay -
2022 Poster: Antibody-Antigen Docking and Design via Hierarchical Structure Refinement »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2022 Spotlight: Learning Stable Classifiers by Transferring Unstable Features »
Yujia Bao · Shiyu Chang · Regina Barzilay -
2022 Spotlight: Antibody-Antigen Docking and Design via Hierarchical Structure Refinement »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2022 Poster: Conformal Prediction Sets with Limited False Positives »
Adam Fisch · Tal Schuster · Tommi Jaakkola · Regina Barzilay -
2022 Poster: EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction »
Hannes Stärk · Octavian Ganea · Lagnajit Pattanaik · Regina Barzilay · Tommi Jaakkola -
2022 Spotlight: Conformal Prediction Sets with Limited False Positives »
Adam Fisch · Tal Schuster · Tommi Jaakkola · Regina Barzilay -
2022 Spotlight: EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction »
Hannes Stärk · Octavian Ganea · Lagnajit Pattanaik · Regina Barzilay · Tommi Jaakkola -
2022 Invited Talk: Solving the Right Problems: Making ML Models Relevant to Healthcare and the Life Sciences »
Regina Barzilay -
2021 : Q&A Contributed Talk »
Jonas Mueller -
2021 : Contributed Talk: Multimodal AutoML on Structured Tables with Text Fields »
Jonas Mueller -
2021 Poster: Few-Shot Conformal Prediction with Auxiliary Tasks »
Adam Fisch · Tal Schuster · Tommi Jaakkola · Regina Barzilay -
2021 Poster: Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers »
Yujia Bao · Shiyu Chang · Regina Barzilay -
2021 Spotlight: Few-Shot Conformal Prediction with Auxiliary Tasks »
Adam Fisch · Tal Schuster · Tommi Jaakkola · Regina Barzilay -
2021 Spotlight: Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers »
Yujia Bao · Shiyu Chang · Regina Barzilay -
2021 Poster: Deep Learning for Functional Data Analysis with Adaptive Basis Layers »
Junwen Yao · Jonas Mueller · Jane-Ling Wang -
2021 Spotlight: Deep Learning for Functional Data Analysis with Adaptive Basis Layers »
Junwen Yao · Jonas Mueller · Jane-Ling Wang -
2021 Poster: Information Obfuscation of Graph Neural Networks »
Peiyuan Liao · Han Zhao · Keyulu Xu · Tommi Jaakkola · Geoff Gordon · Stefanie Jegelka · Ruslan Salakhutdinov -
2021 Spotlight: Information Obfuscation of Graph Neural Networks »
Peiyuan Liao · Han Zhao · Keyulu Xu · Tommi Jaakkola · Geoff Gordon · Stefanie Jegelka · Ruslan Salakhutdinov -
2021 Poster: Learning Task Informed Abstractions »
Xiang Fu · Ge Yang · Pulkit Agrawal · Tommi Jaakkola -
2021 Spotlight: Learning Task Informed Abstractions »
Xiang Fu · Ge Yang · Pulkit Agrawal · Tommi Jaakkola -
2020 : Invited Talk: Tommi Jaakkola »
Tommi Jaakkola -
2020 : 1.2 AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data »
Jonas Mueller -
2020 Poster: Generalization and Representational Limits of Graph Neural Networks »
Vikas K Garg · Stefanie Jegelka · Tommi Jaakkola -
2020 Poster: Multi-Objective Molecule Generation using Interpretable Substructures »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2020 Poster: Invariant Rationalization »
Shiyu Chang · Yang Zhang · Mo Yu · Tommi Jaakkola -
2020 Poster: Predicting deliberative outcomes »
Vikas K Garg · Tommi Jaakkola -
2020 Poster: Hierarchical Generation of Molecular Graphs using Structural Motifs »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2020 Poster: Improving Molecular Design by Stochastic Iterative Target Augmentation »
Kevin Yang · Wengong Jin · Kyle Swanson · Regina Barzilay · Tommi Jaakkola -
2019 Poster: Functional Transparency for Structured Data: a Game-Theoretic Approach »
Guang-He Lee · Wengong Jin · David Alvarez-Melis · Tommi Jaakkola -
2019 Poster: Mixture Models for Diverse Machine Translation: Tricks of the Trade »
Tianxiao Shen · Myle Ott · Michael Auli · Marc'Aurelio Ranzato -
2019 Oral: Mixture Models for Diverse Machine Translation: Tricks of the Trade »
Tianxiao Shen · Myle Ott · Michael Auli · Marc'Aurelio Ranzato -
2019 Oral: Functional Transparency for Structured Data: a Game-Theoretic Approach »
Guang-He Lee · Wengong Jin · David Alvarez-Melis · Tommi Jaakkola -
2018 Poster: Junction Tree Variational Autoencoder for Molecular Graph Generation »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2018 Oral: Junction Tree Variational Autoencoder for Molecular Graph Generation »
Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2017 Poster: Learning Sleep Stages from Radio Signals: A Conditional Adversarial Architecture »
Mingmin Zhao · Shichao Yue · Dina Katabi · Tommi Jaakkola · Matt Bianchi -
2017 Talk: Learning Sleep Stages from Radio Signals: A Conditional Adversarial Architecture »
Mingmin Zhao · Shichao Yue · Dina Katabi · Tommi Jaakkola · Matt Bianchi -
2017 Poster: Sequence to Better Sequence: Continuous Revision of Combinatorial Structures »
Jonas Mueller · David Gifford · Tommi Jaakkola -
2017 Talk: Sequence to Better Sequence: Continuous Revision of Combinatorial Structures »
Jonas Mueller · David Gifford · Tommi Jaakkola -
2017 Poster: Deriving Neural Architectures from Sequence and Graph Kernels »
Tao Lei · Wengong Jin · Regina Barzilay · Tommi Jaakkola -
2017 Talk: Deriving Neural Architectures from Sequence and Graph Kernels »
Tao Lei · Wengong Jin · Regina Barzilay · Tommi Jaakkola