OpenAI released an experimental metalearning approach called Evolved Policy Gradients (EPG), which uses evolutionary algorithms to optimize the loss function of learning agents. This method enables fast training on novel tasks and allows agents to generalize to scenarios outside their training distribution, such as navigating to objects in different locations.
Metalearning
元学习
A method where models learn how to learn, enabling rapid adaptation to new tasks.
Curated Intelligence
This paper provides a survey on first-order meta-learning algorithms. It systematically reviews the theoretical foundations and optimization strategies of mainstream methods like MAML, focusing on improving training efficiency by simplifying second-order derivative computations. Lacking specific experimental data or new model releases, its core value lies in organizing theoretical frameworks rather than technical breakthroughs.
OpenAI researchers introduced Reptile, a simple and scalable meta-learning algorithm. It works by repeatedly sampling tasks, performing stochastic gradient descent, and updating initial parameters towards the final learned parameters. Reptile requires only black-box access to optimizers like SGD or Adam, offering computational efficiency and performance similar to first-order MAML but with greater simplicity.
In 2017, OpenAI published research on 'Meta-learning for wrestling,' demonstrating the application of meta-learning algorithms to achieve rapid policy adaptation in multi-agent adversarial environments. The study validated that agents could quickly adjust their behavior in response to opponent changes within simulated settings, providing early empirical support for generalization and adaptability in reinforcement learning.
OpenAI researchers proposed a meta-learning framework called 'one-shot imitation learning,' enabling robots to learn new tasks from very few demonstrations and instantly generalize. The method uses a neural network taking a reference demonstration and current state as input, trained on large task sets to reduce reliance on feature engineering and massive samples. This work offers a new technical path for autonomous skill acquisition in few-shot robotics scenarios.
OpenAI introduces RL², a framework that encodes a 'fast' reinforcement learning algorithm into the weights of a Recurrent Neural Network (RNN). By training this RNN with a general-purpose 'slow' RL algorithm across many tasks, it gains cross-episode memory, enabling rapid adaptation to new environments. Experiments demonstrate effectiveness on multi-armed bandits and large-scale visual navigation.