ALL INTELLIGENCE
OpenAI 官方博客T1·Models

OpenAI Publishes Variational Lossy Autoencoder Paper

Original title: Variational lossy autoencoder
45AI Score

OpenAI published the 'Variational Lossy Autoencoder' research paper in 2016, introducing a generative model architecture that combines variational inference with lossy compression. The model achieved state-of-the-art density estimation results on datasets like MNIST and OMNIGLOT, laying important technical groundwork for subsequent sequence modeling and generative AI.

Full Report

Contents2 sections

Skip to main content

Log inTry ChatGPT(opens in a new window)

Try ChatGPT(opens in a new window)Login

OpenAI

November 8, 2016

Publication

Variational lossy autoencoder

Read paper(opens in a new window)

Image 1: Variational Lossy Autoencoder

Loading…

Audio 2

Share

Abstract

Representation learning seeks to expose certain aspects of observed data in a learned representation that’s amenable to downstream tasks like classification. For instance, a good representation for 2D images might be one that describes only global structure and discards information about detailed texture. In this paper, we present a simple but principled method to learn such global representations by combining Variational Autoencoder (VAE) with neural autoregressive models such as RNN, MADE and PixelRNN/CNN. Our proposed VAE model allows us to have control over what the global latent code can learn and , by designing the architecture accordingly, we can force the global latent code to discard irrelevant information such as texture in 2D images, and hence the VAE only "autoencodes" data in a lossy fashion. In addition, by leveraging autoregressive models as both prior distribution p(z) and decoding distribution p(x|z), we can greatly improve generative modeling performance of VAEs, achieving new state-of-the-art results on MNIST, OMNIGLOT and Caltech-101 Silhouettes density estimation tasks.

Authors

Xi Chen, Durk Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, Pieter Abbeel