ALL INTELLIGENCE
OpenAI 官方博客T1·Agents

OpenAI Releases Video PreTraining (VPT) Model to Master Minecraft Crafting with Minimal Labels

Original title: Learning to play Minecraft with Video PreTraining
78AI Score

OpenAI introduces the Video PreTraining (VPT) model for Minecraft, trained on massive unlabeled human gameplay videos and a small set of labeled contractor data. The agent operates via native keyboard and mouse interfaces and successfully crafts diamond tools—a complex task requiring over 20 minutes and 24,000 actions for proficient humans. This demonstrates the potential of leveraging unstructured visual data to build general computer-using agents.

Full Report

We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data. With fine-tuning, our model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Our model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents.