OpenAI 发布视频预训练模型 VPT:仅用少量标注数据即可在 Minecraft 中合成钻石工具

原标题:Learning to play Minecraft with Video PreTraining
78AI 研判分

OpenAI 推出基于视频预训练(VPT)的 Minecraft 智能体,通过大规模无标签人类游戏视频和极少量标注数据进行训练。该模型能够直接操作键盘鼠标界面,成功完成需人类专家耗时20分钟、执行24000次动作才能完成的“合成钻石工具”任务。这一成果展示了利用非结构化视觉数据构建通用计算机使用智能体的可行性。

报道正文

We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data. With fine-tuning, our model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Our model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents.