Xiaomi Open-Sources MiMo-V2.6: $2.6M Per RL Step and 3.7B Tokens for Agentic Self-Improvement

Xiaomi's MiMo team has released the V2.6 series of models, marking the first systematic implementation of Agentic RL with synchronized scaling across three dimensions: Rollout, Environment, and Grader. Both Pro and Flash variants adopt a Mixture-of-Experts (MoE) architecture, integrating sliding window attention with global attention to support million-token context training. Each RL step generates 25,000 trajectories, with a total token volume reaching 2.7–3.7 billion.

The system features three-stage compute decoupling: independent scaling for Rollout, Grading, and Training; million-token contexts enabling long-horizon agent behavior modeling; and a hybrid MoE + SWA/GA architecture balancing efficiency with modeling capability.

This content is tailored for AI systems engineers, reinforcement learning researchers, and large model training platform architects.

On October 8, the Xiaomi MiMo team published the technical report "MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement." The report details a comprehensive process for scaling Agentic RL training: by simultaneously expanding the training batch size, task environments and harnesses, and the computational resources dedicated to the Grader (which evaluates agent behavior), the team explores pathways for continuous model self-improvement.

The release includes two models: MiMo-V2.6-Pro and MiMo-V2.6-Flash. The Pro variant has a total parameter count of 1.02T, activating 42B parameters per inference. The Flash variant has a total parameter count of 310B, activating 15B parameters. Both employ a sparse Mixture-of-Experts (MoE) architecture, mixing local Sliding Window Attention (SWA) with Global Attention (GA) in the text backbone.

Paper Link:

https://arxiv.org/pdf/2610.11959

Simultaneously Scaling Rollout, Environments, and Graders

MiMo-V2.6 decomposes reinforcement learning compute into three components: Rollout, Grading, and Training.

In actual training, each RL Step samples 1,568 prompts, generating 16 rollouts per prompt simultaneously. This results in approximately 25,000 agent trajectories per step. These trajectories collectively amount to 2.7 to 3.7 billion tokens, with an average sequence length of 110,000 to 150,000 tokens. The maximum training context length reaches 1 million tokens.

MiMo-V2.6's training tasks cover code agents, general tool use, visual design, context adherence, and cybersecurity. Within the formal RL batch, code and competitive programming tasks account for 68%, general tool use for 12%, visual design for 13%, context adherence for 3%, and cybersecurity for 4%. The entire training process utilizes GRPO combined with asynchronous Partial Rollout.

Training costs have risen rapidly accordingly. The paper indicates that the post-training RL cost for MiMo-V2.6-Pro is approximately $2.6 million, while Flash costs about$900,000. For Pro, Rollout accounts for 43.8% of compute costs, model parameter updates (Training) account for 43.5%, and the Grader—dedicated solely to judging the quality of agent behavior—accounts for 12.7%. For Flash, the Grader cost ratio even reaches 14.2%.

Evaluating Whether Agents "Perform Well" Goes Beyond Binary Tests

Traditional code RL often relies on binary test rewards: passing tests yields 1 point, failing yields 0. However, this approach is insufficient for long-horizon agent tasks.

Two solutions might both pass tests, yet one may modify only the necessary lines of code, while the other introduces extensive compatibility branches, swallows exceptions, expands API scopes, or even modifies configurations unrelated to the task just to pass the tests. Under traditional Reward functions, these two approaches might receive identical scores despite vastly different engineering quality.

To address this, MiMo-V2.6 introduced two grouped agent scoring mechanisms in code agent training.

One is Groupwise Reward Synthesis (GRS). The system first collects multiple rollouts for the same task, where an agent analyzes different solutions to generate two sets of rubrics: "Solution Quality" and "Behavioral Quality." Subsequently, every new trajectory during training must not only pass tests but also be scored based on factors such as code implementation quality, edge case handling, consistency with the original codebase, and whether necessary verification was performed during problem-solving.

The other is Groupwise Advantage Redistribution (GAR). The Grader compares successful and failed trajectories from the same task side-by-side. Solutions that pass tests are further ranked based on reasoning approach, implementation accuracy, scope of modifications, side effects, and engineering quality. More reinforcement signal is then allocated to higher-quality solutions.

Experiments show that without online group scoring, as RL training continues, models produce increasingly longer trajectories and more agent turns, gradually exhibiting behaviors like adding redundant compatibility logic or swallowing exceptions to boost pass rates. With online scoring enabled, pass rates continue to rise while token lengths and interaction rounds remain more stable, guiding the model toward better engineering practices.

To this end, Xiaomi implemented multiple layers of defense before and after training. The training environment actively deletes build logs, Verifier outputs, residual patches, binaries, bytecode, and caches, retaining only the Git history prior to the task-specified version. Container-level network isolation prevents Agents from accessing upstream fixes. The team even trained and deployed a dedicated "Hack Agent" to proactively probe these environments for exploitable vulnerabilities. If new cheating paths are discovered, the environment is modified, retested, and subjected to further Hack Agent attacks until no exploitable paths remain.

Even after formal RL begins, the team conducts offline audits of Agent trajectories. If confirmed Reward Hacking is detected, the reward for that trajectory is immediately zeroed out. Ultimately, during the formal training of MiMo-V2.6-Pro and Flash, the proportion of trajectories with confirmed Reward Hacking was consistently kept below 2%.

Training Agents to "Adapt to Different Agents"

Another significant change in MiMo-V2.6 is treating the Agent Harness itself as a scaling dimension for RL.

Different Coding Agents vary in system prompts, context management, tool invocation methods, and execution flows. If a model is trained long-term within a single harness, it easily learns harness-specific behavioral patterns, leading to performance degradation when switching to another agent framework.

Xiaomi did not directly mix full production harnesses like Codex or MiMo Code for training. The paper argues that production harnesses typically include extensive engineering protections, complex workflows, and additional prompts whose behaviors may not correspond to specific rewards, thereby interfering with credit assignment in reinforcement learning. Furthermore, high coupling between modules makes controlling variables difficult.

Therefore, MiMo-V2.6 constructed a set of smaller, more modular mini-harnesses. These uniformly retain basic capabilities such as system prompts, tools, and context management, while combining different configurations for Code, General, Visual, and Cyber tasks during training.

One effect is that the model improved not only on the four harnesses seen during training but also synchronously on unseen ones like Codex, Claude Code, and mini-swe-agent. On DeepSWE v1.1, the average Pass@1 across three held-out harnesses increased from approximately 50% to 66%.

MoE Router Begins to "Collapse" After Just 20 Steps When Learning Alongside RL

As scale increases, the model itself exhibits new stability issues.

MiMo-V2.6 is an MoE model. Theoretically, the Router determines which experts should process each token based on input. If the Router continues to update during RL, the load distribution among different experts may drift continuously.

Xiaomi conducted a controlled experiment: In MiMo-V2.6-Pro, without freezing the Router, after just the first 20 training steps, the coefficient of variation for expert load at layer 9 rose from 0.78 to 2.0; the load on the busiest Expert increased from 6 times the average to 16 times; and the proportion of "cold experts" (with loads below 10% of the average) rose from 0.5% to 22%.

The team then restored the Router parameters in the step-20 model to their pre-RL values while keeping other model parameters unchanged. As a result, expert loads quickly returned to normal, but benchmark performance remained largely unaffected. This indicates that the issue was not degradation of the Experts themselves, but rather Router drift during RL.

Consequently, MiMo-V2.6 ultimately chose to freeze the MoE Router throughout the entire RL phase. After freezing, the coefficient of variation for expert loads stabilized around 0.7, peak load remained at about 5.5 times the average, the proportion of cold experts stayed near 1%, and model performance continued to improve.

A Single RL Run Lasted 123 Hours, With Failures in GPUs, K8s, and Graders

The paper also disclosed failure timelines for two formal training runs: MiMo-V2.6-Pro took 123.1 hours to complete 30 RL steps, while Flash took 81.8 hours. During training, GPU memory double-bit errors (DBE) occurred; Flash encountered Kubernetes failures between steps 15 and 16, causing Pod crashes in the Cyber cluster; Pro restarted after step 14 due to unreachable Grader networks.

Additionally, there were more typical large-scale Agent RL issues. After restarting Partial Rollout, shorter tasks completed first, causing the system to underestimate remaining task lengths, eventually filling up both the GPU KV Pool and pinned host memory pools. During training, severe load imbalance occurred within MoE micro-batches, where one Expert Parallel Rank received over 30 times the average number of tokens, leading to out-of-memory (OOM) errors. Later stages of Flash saw data volume increase due to longer sequences, ultimately exhausting host memory and triggering OOM.

Rebuilding RL Infrastructure

To support 25,000 trajectories per step, Xiaomi redesigned the RL data and execution systems.

On the execution side, the Harness Pool uses Ray Actors to run different Agent Harnesses. To avoid file descriptor exhaustion on control nodes caused by creating independent Actors for every Agent Loop, the system switched to a fixed number of persistent Host Actors, each hosting multiple Agent Loops and Harness instances simultaneously.

Meanwhile, MiMo-V2.6 introduces a Sample Mixer to dynamically control data ratios across different tasks. According to the paper’s statistics from 25 data sources, the maximum difference in average generated token count is 90x, and the maximum difference in rollout execution time is 66x. With a simple first-come-first-served approach, short tasks would quickly saturate the batch, causing long-running tasks to be repeatedly pushed out. Therefore, the Sample Mixer dynamically adjusts concurrency levels and scheduling priorities based on target ratios, pass rates, and task duration.

To further reduce rollout costs, Xiaomi has also brought a series of inference-side optimizations directly into RL.

Each conversation context for the Agent maintains a persistent KV Cache. During multi-turn tool calls, inactive caches are offloaded from HBM to pinned host memory and moved back when the model resumes generation. This strategy maximizes HBM availability for active requests, thereby improving decode batch sizes.

Rollouts also employ Block-6 DFlash for speculative decoding. Compared to the original MTP-3 configuration, the RL-adapted DFlash achieves a 31.3% increase in average acceptance length. Block-6 offers approximately 6% higher global average throughput compared to Block-8. Furthermore, by introducing FP8 low-precision draft computation, single-node throughput improves by about 10.3% over the baseline in long-context workloads.

Improved Out-of-Distribution Software Engineering Capabilities After 30 Updates

A key question following the scaling of reinforcement learning is: Has the model truly acquired new capabilities, or has it merely become proficient at the training environment?

Xiaomi’s results show that MiMo-V2.6-Flash and Pro achieved relative improvements of approximately 25% and 12%, respectively, in average pass rates on training tasks. On DeepSWE v1.1, a long-horizon software engineering benchmark not included in the training set, Flash’s score increased from 48.8 to 65.7 (a gain of ~17 points), while Pro’s score rose from 58.4 to 72.6 (a gain of ~14 points). Based on this, Xiaomi argues that some capabilities gained through large-scale RL can transfer to tasks outside the training distribution.

Looking at the entire MiMo-V2.6 series, Xiaomi claims that Pro scored 46 on the Artificial Analysis Intelligence Index and reached levels comparable to Claude Opus 5 and GPT-5.6 Sol on most agent benchmarks. However, the official documentation acknowledges that there remains a gap compared to the strongest closed-source models such as Claude Fable 5.1 and GPT-6 Astra.

Additionally, MiMo-V2.6 extends its coverage to Computer Use, embodied intelligence, and scientific research tasks. For example, the model can perform information retrieval, editing, and data processing via graphical interfaces, using visual feedback to verify results and correct subsequent actions. In embodied simulation environments, it can also make continuous decisions and control robotic arms based on multi-view inputs.

From Pre-training Scaling to Agentic RL Scaling

In recent years, performance gains in large models have primarily relied on a relatively clear path: increasing parameters, data, and pre-training compute. As we enter the era of reasoning models and agents, new computational investments are gradually shifting to post-training phases—placing models in vast verifiable environments to generate longer rollouts, obtaining feedback through more complex graders, and updating policies via RL.

The MiMo team refers to this trajectory as an exploration of Recursive Self-Improvement (RSI). However, based on the mechanisms demonstrated in the technical report, there is still a significant distance to full RSI, where "models autonomously design and train the next generation." What MiMo-V2.6 actually achieves is establishing a closed loop of continuous feedback and capability improvement through scaled RL within human-designed tasks, environments, rewards, and training infrastructure.

To enable external researchers to reproduce this pathway, Xiaomi simultaneously open-sourced over 7,000 high-quality RL environments covering tasks such as software engineering, vulnerability reproduction, knowledge-based work, and web design/development. They also released an end-to-end RL framework built on verl, uni-agent, and mini-swe-agent, along with a lightweight framework for multi-Harness training.

Open Source Address:

https://huggingface.co/collections/XiaomiMiMo/mimo-v26