OpenAI Publishes Variational Lossy Autoencoder Paper
OpenAI published the 'Variational Lossy Autoencoder' research paper in 2016, introducing a generative model architecture that combines variational inference with lossy compression. The model achieved state-of-the-art density estimation results on datasets like MNIST and OMNIGLOT, laying important technical groundwork for subsequent sequence modeling and generative AI.
Related Entities
Full Report
Contents2 sections
Log inTry ChatGPT(opens in a new window)
- Research
- Products
- Business
- Developers
- Company
- Foundation(opens in a new window)
Try ChatGPT(opens in a new window)Login
OpenAI
November 8, 2016
Variational lossy autoencoder
Read paper(opens in a new window)

Loading…
Share
Abstract
Representation learning seeks to expose certain aspects of observed data in a learned representation that’s amenable to downstream tasks like classification. For instance, a good representation for 2D images might be one that describes only global structure and discards information about detailed texture. In this paper, we present a simple but principled method to learn such global representations by combining Variational Autoencoder (VAE) with neural autoregressive models such as RNN, MADE and PixelRNN/CNN. Our proposed VAE model allows us to have control over what the global latent code can learn and , by designing the architecture accordingly, we can force the global latent code to discard irrelevant information such as texture in 2D images, and hence the VAE only "autoencodes" data in a lossy fashion. In addition, by leveraging autoregressive models as both prior distribution p(z) and decoding distribution p(x|z), we can greatly improve generative modeling performance of VAEs, achieving new state-of-the-art results on MNIST, OMNIGLOT and Caltech-101 Silhouettes density estimation tasks.
Authors
Xi Chen, Durk Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel