ByteDance and Duke Unveil Axion 1.0 with Two-Stage Agent Training

Original title: 训练一个强大的智能体,究竟有多简单?字节跳动等推出Axion 1.0
78AI Score

A team from Duke University, ByteDance, the University of Maryland and SUNY Buffalo released Axion 1.0, which reduces agent training to two stages: supervised fine-tuning and reinforcement learning, dropping mid-training, DPO and on-policy distillation. Axion 1.0-122B scores 59.0% on the restricted-access Humanity's Last Exam, above GPT-6, while the 35B variant reaches 62.2% on SEAL-0. The team open-sourced data construction, filtering, verification, prompts and training code.

Full Report

It can search for information, run code, and even complete multi-step scientific research tasks—how many stages does it actually take to train such an agent?

Mid-training, supervised fine-tuning, preference optimization, online policy distillation, reinforcement learning… As agent capabilities continue to grow, the training pipeline has become increasingly long. With each additional stage, researchers need to prepare the corresponding data, coordinate new training objectives, and then tune another set of training parameters.

A research team from Duke University, ByteDance, the University of Maryland, College Park, and the University at Buffalo, SUNY, has introduced Axion 1.0. Its training pipeline consists of only two stages: supervised fine-tuning (SFT) and reinforcement learning (RL), with no additional mid-training, direct preference optimization (DPO), or online policy distillation (OPD).

With this streamlined training pipeline, Axion 1.0 demonstrates strong scientific reasoning and tool-use capabilities. According to the paper, Axion 1.0-122B achieves 59.0% on the full Humanity's Last Exam (HLE) benchmark with access to some websites restricted (surpassing GPT-6), and 66.7% on FrontierScience Research; the 35B version achieves 62.2% on the information-seeking benchmark SEAL-0.

More importantly, the team has fully open-sourced the data construction, generation, filtering, verification, prompts, and training code, disclosing the complete training methodology to the community. The concise three-stage pipeline and transparent open-source implementation demystify the training of agent foundation models and enable more people to train their own powerful agent models.

Method Introduction

The key to Axion 1.0's success lies in constructing high-quality training data tailored to the different capabilities required by agents. Specifically, Axion 1.0 divides agent training tasks into two categories: one involves reasoning and computation based on the complete information provided in the problem, and the other requires using tools to obtain missing external information and then combining the evidence to complete the reasoning. The two SFT stages focus on these two types of tasks respectively, enabling the model to first improve its reasoning ability and then learn how to acquire and utilize external information.

Step 1: Add challenges to problems the model can already solve

The first stage uses problems from mathematics, physics, chemistry, biology, and computer science to train the model's multi-step reasoning and computation abilities. However, the base model can already answer many public problems correctly. If we continue training on these problems, how can we provide more learning space? Axion 1.0's approach is to increase problem difficulty in a targeted manner based on the base model's performance.

The research team first enhances the difficulty of seed problems: the answer to the original problem becomes an intermediate quantity in the new problem, adding a layer of dependency to the solution. Then the base model attempts the new problems: problems it answers incorrectly enter the candidate pool; problems it answers correctly, along with their successful solutions, are passed to a generative model to further increase difficulty.

For example, if the seed problem asks for the sum from 1 to 10, the new problem could replace the fixed values with parameters and ask for a general expression; or it could expand the range of values so that the original enumeration strategy no longer works. Other rewriting methods include adding reasoning that depends on the result of a previous step, introducing conditions that require case-by-case discussion, or even combinations of these strategies.

After each rewrite, the reference answer is re-verified, and problems with insufficient conditions or ambiguous answers are removed. Correct solution trajectories are then collected to form approximately 10,800 first-stage SFT samples. This stage allows the use of Python but does not enable search or web access, so the model first learns to handle reasoning and computation within the problem itself.

Step 2: Make the model go out and find answers

In addition to strong internal reasoning ability, when an agent model encounters a problem that cannot be solved using internal knowledge, it also needs to determine what information is missing, where to look for it, and how to use the retrieved content for reasoning. The task construction in the second stage is centered around these capabilities.

Axion 1.0 starts from public structured data, uses programs to perform filtering, select a unique entity, and compute the answer, then has a language model write this process into a natural language question. The reference answer is already determined when the question is created, making it easy to later verify whether the model truly solved the problem correctly.

The paper provides an intuitive example: among the solar system planets with planetary rings, find the one with the smallest mass, then calculate its average density. The model needs to first identify Uranus, look up its mass and radius, and finally complete the calculation. One question thus links entity recognition, fact retrieval, and numerical computation.

When collecting trajectories, the database tables and reference answers used for question creation are not directly provided to the model; the model needs to obtain the required information through search, web access, and Python. Questions that can be answered correctly without retrieval are also filtered out. The program also checks whether entity selection is unique, whether parameters are complete, and whether the natural language rewriting omits conditions or leaks the answer. Ultimately, the second stage retains approximately 7,500 correct tool-interaction trajectories. Both stages of SFT preserve the full interaction context, and training loss is computed only on the reasoning, tool calls, and answers generated by the assistant.

Step 3: Continue learning in real tool interactions

With demonstrations of reasoning and tool use, the 35B and 122B main models further use asynchronous GRPO for reinforcement learning, generating their own problem-solving trajectories in search, web access, academic retrieval, and Python environments.

The most critical design in the RL stage is the staged reward mechanism. And this implementation depends on the design of the earlier data construction. Programmatic question creation can provide both the final answer and record the entities and numerical values that should be obtained during the problem-solving process. Therefore, the model receives full reward when it ultimately answers correctly; when it does not answer correctly, verified intermediate progress can still receive partial reward, while a penalty is applied when no final answer is submitted.

Intermediate progress also has sequential dependencies. For example, first select the correct entity, then obtain parameters, and finally calculate; subsequent steps can only count toward progress when the preceding steps pass inspection. Entity recognition that requires external evidence must also be supported by tool observations. In this way, correct content appearing incidentally in retrieval results will not be directly treated as the model having completed the reasoning.

Experimental Results

The two main models of Axion 1.0 are based on Qwen3.6-35B-A3B and Qwen3.5-122B-A10B, respectively. The team observed their scientific reasoning, information seeking, and tool use capabilities across six evaluations.

Main Results

  • HLE: Axion 122B achieves 59.0%, higher than GPT-6 Astra (57.2%) and GPT-5.5 (52.2%) listed in the paper, and close to Claude Opus 5 (63.6%) and Claude Fable 5 (63.8%).
  • FrontierScience Research: Axion 122B achieves 66.7%, surpassing GPT-5.6 Sol (63.3%) and also higher than GPT-5.5 (26.7%).
  • FrontierScience Olympiad: The 122B and 35B versions achieve 87.0% and 86.0%, respectively, both exceeding GPT-5.6 Sol (79.0%) and Gemini 3.1 Pro (85.0%).
  • SEAL-0: The 35B version achieves 62.2%, surpassing GPT-5.6 Sol (53.2%), Gemini 3.1 Pro (53.2%), and Agents-A1 (56.4%).
  • HiPhO: The 122B version achieves 46.2%, higher than GPT-5.6 Sol (42.9%) and GPT-5.5 (43.3%).

Effects of Each Training Stage

Taking the 35B model as an example, after the first stage of SFT, FrontierScience Olympiad improves from 71.0% to 83.0%, reflecting enhanced scientific reasoning ability; after the second stage introduces tool-use training, BioMysteryBench improves from 17.6% to 35.3%, and FrontierScience Research improves from 48.3% to 58.3%; RL further improves SEAL-0 from 53.2% to 62.2%. The results of consecutive training checkpoints demonstrate the model's capability improvement process from reasoning and information acquisition to multi-step tool interaction.

Different Base Models

The same two-stage SFT method was applied to Qwen3.5-35B-A3B and Qwen3.6-35B-A3B. Before training, the two differed by 9.2 percentage points on HLE-130; after training, the gap narrowed to 0.8 percentage points, and both reached 35.3% on BioMysteryBench. These results indicate that the Axion 1.0 training method can be applied to different base models, helping models with weaker starting points significantly narrow the performance gap.

Model Scale

The Axion 1.0 training method was also applied to 4B and 9B models. After only two-stage SFT, the two small models achieve 53.1% and 63.1% on HLE-130, respectively, and reach 81.0% and 83.0% on FrontierScience Olympiad, respectively; the 9B model also achieves 55.0% on FrontierScience Research. This shows that the constructed training data is equally effective on smaller models.

Returning to the initial question: how simple is it, after all, to train a powerful agent? Axion 1.0 demonstrates a feasible path consisting only of two-stage SFT and RL. What supports it is designed and verified learning tasks: using the model's existing solutions to guide difficulty progression, using public data to construct questions that require external evidence, and then continuing optimization through rewards in tool interactions.

The team has released the data construction, training, and testing code. For researchers who wish to reproduce and improve agent training, these materials provide a concrete starting point.

Author Team

The author team comes from Duke University, ByteDance, the University of Maryland, College Park, and the State University of New York at Buffalo.

Image

© THE END

For reprints, please contact this official account for authorization.

Submissions or press inquiries: [email protected]

Open in WeChat