OpenAI Releases GPT-2: Unsupervised Pre-training Enables Multi-task Generalization
Original title: Better language models and their implications
95AI Score
OpenAI introduced GPT-2, a large-scale unsupervised language model achieving state-of-the-art performance across benchmarks. It performs reading comprehension, translation, QA, and summarization without task-specific training, establishing the pre-training paradigm for general-purpose AI.
Full Report
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.