This study demonstrates that large transformer models trained on pixel sequences can generate coherent image completions and samples, analogous to language generation. By correlating sample quality with classification accuracy, the research shows that the best generative model features compete with top convolutional nets in unsupervised settings.
Unsupervised Learning
无监督学习
Machine learning method that learns features from intrinsic data structure without human-labeled data.
Curated Intelligence
OpenAI introduced MuseNet, a deep neural network capable of generating 4-minute compositions with 10 instruments and blending styles from country to Mozart. The model uses the same large-scale transformer architecture as GPT-2, learning harmony, rhythm, and style patterns by predicting the next token in hundreds of thousands of MIDI files without explicit musical programming.
OpenAI introduced GPT-2, a large-scale unsupervised language model achieving state-of-the-art performance across benchmarks. It performs reading comprehension, translation, QA, and summarization without task-specific training, establishing the pre-training paradigm for general-purpose AI.
OpenAI introduced an unsupervised system trained solely to predict the next character in Amazon reviews. The model learned a robust representation of sentiment, demonstrating that language models can capture semantic nuances without labeled data.
OpenAI published a review article on generative models, introducing four related projects and explaining the definition, importance, and future directions of these unsupervised learning techniques. As early content from 2016, it focuses primarily on conceptual education rather than specific technical breakthroughs or new model releases.