FireFlash Trains 300B Agent on 64 B300 GPUs in Under Five Days

FireFlash Releases the Complete Recipe for Long-Horizon Agentic RL

Long-horizon agents are moving from "fixing a single bug" toward hours-long terminal and coding work, but their reinforcement learning training is also exceptionally expensive: frontier labs can scale RL across thousands of GPUs, whereas for most teams a single training sample is often a complete attempt lasting up to 3 hours with hundreds of tool calls—any misstep at any stage wastes hours of GPU time.

Recently, researchers Wenbo Hu, Nanyun Peng, Dakuo Wang, Kai-Wei Chang, and others published FireFlash on the Agent Economics blog: a 300B-class terminal and coding agent trained via full-parameter reinforcement learning starting from GLM-5.3-Flash.

FireFlash achieves 46.2% Pass@1 on Terminal-Bench 4.0, on par with Fable 5 and GPT-6 Sol, while costing more than 30× less per task than Fable 5 at its highest reasoning effort; this improvement also transfers to two software engineering benchmarks, DeepSWE v1.1 and SWE-Bench Pro Verified. The entire training run used only 64 NVIDIA B300 GPUs for under 5 days, and the team has released the complete training recipe.

• Blog title: FireFlash: An Open Recipe for Long-Horizon Agentic RL Achieving the Performance-Cost Pareto Frontier
• Blog URL: https://agenteconomics.org/posts/agentic-rl-recipe
• Authors: Wenbo Hu, Jesse Cai, Po-nien Kung, Yuxuan Lu, Di Wu, Huimin Zeng, Nanyun Peng, Dakuo Wang, Kai-Wei Chang

Figure 1: Pass@1 versus average per-task cost (log scale) on Terminal-Bench 4.0. The dashed line is the Pareto frontier, and the arrow shows FireFlash's improvement over GLM-5.3-Flash. Since the model is self-hosted by the team, FireFlash is billed at GLM-5.3-Flash's public pricing.

Table 1: Pass@1 on three benchmarks (averaged over 5 trials). GLM-5.3-Flash figures are its official results on each benchmark; FireFlash is evaluated in the team's own harness.

A Hundred-GPU-Scale Cluster,

How to Reach the Pareto Frontier?

The team performed full-parameter RL starting from GLM-5.3-Flash (321B total parameters, 17.4B activated per token), training all 320B language parameters (including the MTP layers) while freezing the vision tower and DSA indexer. They argue that a hundred-GPU-scale cluster can also achieve frontier-level training efficiency, with three keys: carefully curated agentic RL environments so that every task provides reliable signal at an appropriate difficulty; an RL recipe tailored for long, stale, low-precision rollouts; and a coding harness (agent runtime framework) shared across model optimization, training, and evaluation. Specifically, the team revisited and optimized five key stages of the post-training pipeline:

① Infrastructure: GPUs don't wait for the slowest attempt. Training and rollout each use a dedicated set of GPUs, rollouts up to 6 weight versions stale are allowed for training, and MTP accelerates decoding by 1.38×; each attempt runs in an isolated microVM, reducing rollout groups lost to infrastructure failures from 12% to 0.5%.

② Task curation: every rollout group carries signal. Only tasks with an initial pass rate between 15% and 85% are kept (measured under conditions identical to training with at least 4 samples), reducing the share of zero-signal rollout groups from 48% to 10.2%.

③ Reward: score only genuine solutions. Each task must pass nine validation gates before entering the pool, and a calibrated final partial score is used; an efficiency bonus for shorter solutions cuts the number of tokens per solve to 0.84× the original without sacrificing accuracy.

④ Harness: long-horizon tasks fit in context, and training matches evaluation. Context compression, redirecting output to files, and background execution reduce per-attempt cost by about 33%; training and evaluation share the same harness, which alone puts GLM-5.3-Flash 4.4 points above Claude Code before any RL.

⑤ RL objective: stale FP8 samples can drive learning without drift. GRPO combined with truncated importance sampling, top-k score centering, success shifting, and a loss floor remains stable without KL penalties or entropy bonuses, with only 1%–3% of tokens truncated.

Infrastructure:

Fully Asynchronous RL

Core Lesson 1: Idle GPUs cost more than stale rollouts—let sampling and training each run at full speed.

Separate GPUs for training and sampling. Half the GPUs train with the slime framework in BF16, while the other half run the SGLang inference engine, with both weights and KV cache in FP8. After each training step, the new weights are synchronized to the rollout engine via NCCL, so training and sampling do not block each other (Figure 2).

Train on stale rollouts. Because sampling runs continuously, a given turn may be generated by weights that lag the trainer by several steps; it is discarded only when it comes from weights more than 6 versions old. The bias introduced by policy staleness is small: the log-probability gap between the trainer and the rollout is 0.065 at 1 version of staleness, and rises only to 0.08 at 4–6 versions of staleness.

Fast, isolated rollouts. The model's native multi-token prediction (MTP) head is used for speculative decoding, raising decoding speed by 1.38x. Every rollout and scoring run executes in a separate Firecracker microVM via AgentENV, isolating attempts from one another and reducing the share of rollout groups lost to infrastructure errors from 12% to 0.5%.

Figure 2: Fully asynchronous RL. Rollouts and training each occupy half the GPUs and never wait on each other. The inference engine samples continuously, with each attempt running in a separate microVM forked (in under 100 ms) from its task's parent VM; completed groups queue up for the trainer to batch. Groups in which all attempts receive the same score are discarded, and turns sampled from weights more than 6 versions old produce no gradient. After each update, the trainer pushes the new weights to the inference engine.

Task filtering:

Train only where the model is "unsure"

Core Lesson 2: Spend rollouts only on tasks the model is uncertain about, and measure that uncertainty under exactly the same conditions as training.

GRPO learns from the relative differences among multiple attempts at the same task. A task the model can always solve, or can never solve, consumes a full group of rollouts while providing zero learning signal. The team therefore probed each candidate task with at least 4 rollouts using the base model, keeping only tasks with an initial pass rate between 15% and 85%. The probing strictly followed the training configuration—the same harness, reasoning effort, context window, turn budget, and sandbox—because a task's pass rate depends on every one of these. This filtering reduced the share of zero-signal rollout groups from 48% to 10.2%. In the final training data, long-horizon agentic tasks account for about 52% and software engineering tasks for about 48% (Figure 3).

Figure 3: Training data composition. Orange denotes long-horizon agentic tasks (52% total): the team's in-house long-horizon agentic tasks (30%) and their hard tier (11%), long-horizon coding skills (9%), and the public MiMo-TB and Tasksmith (1% each); blue denotes software engineering tasks (48% total): the public MiMo-code (18%), and the team's in-house SWE tasks (18%, predominantly Python, and 12% covering TypeScript, Go, Python, JavaScript, and Rust).

Graders and rewards:

Validate the grader before worrying about reward hacking

Core Lesson 3: Tasks the model never solves are often broken tasks—validate every grader before the reward reaches the model.

When reviewing tasks that were never solved, the team found that nearly half were "broken" rather than "hard": the problem itself was flawed, it relied on fragile tests that misjudge correct solutions, or it hid a "protected file" trap. Some graders awarded partial credit merely for matching the output format, or crashed on realistic agent outputs. To eliminate these problems, every new task must pass nine validation gates (Figure 4) before entering the training pool, confirming that the task is solvable as stated, that the sandbox is closed and stable, and that the grader's scores are deserved.

Figure 4: The nine gates a new task must pass before entering the training pool. Blue denotes the task (2 gates): a reference solution passes in the training sandbox through the exact same training pipeline; every requirement the grader checks is stated in the problem. Green denotes the sandbox (3 gates): it runs offline; it starts reliably every time without hanging; the environment does not leak answers. Orange denotes the grader (4 gates): attempts that do nothing or submit only boilerplate score 0; tests are not fragile, and any correct solution passes; even malformed real outputs can be scored without crashing; identical outputs always receive identical scores.

Reward design. Each attempt is scored once at the end, with a terminal reward in [0, 1]. A full solve earns 1.0; incomplete attempts receive partial credit for the validation checks they pass, and software engineering tasks are scored by the failing tests they fix. Solved attempts can also earn a length bonus of up to 0.2, scaled by how many fewer tokens they use relative to the longest solution in the same group. This shaping reduced the number of tokens per solve to 0.84x of the original, with no drop in evaluation accuracy.

Harness:

Built for long-horizon attempts, with training and evaluation fully aligned

Core Lesson 4: The harness is part of the model—whichever one you evaluate with, train with the same one.

The team's harness follows Claude Code conventions: four tools (Bash, Read, Write, Edit), native tool calling, retained reasoning history, and context compaction. Crucially, training rollouts and evaluation trials are driven by exactly the same harness, with automated unit tests ensuring the two stay consistent in code and prompts. Even before any RL, this harness gave GLM-5.3-Flash a 4.4-point improvement on Terminal-Bench 4.0 over its official Claude Code score (Table 2).

Table 2: Performance of the same model, GLM-5.3-Flash, on Terminal-Bench 4.0 using Claude Code (official leaderboard results) and the team's harness, respectively. Standard time limit, with per-trial cost and per-solve cost calculated at public pricing.

Long-horizon attempts must also be affordable. State-preserving context compaction triggers when occupancy reaches 85% of the context window, preserving the prefix cache by appending a sectional summary prompt to the end of the existing conversation history; commands write stdout to a file and return only a truncated preview with the filename; long-running commands are moved to background asynchronous execution rather than being terminated on timeout. Together, these features reduce per-trial cost by about one third. In addition, a prompt specification requiring an efficient workflow (in-place editing, writing everything in one pass, no sleep polling) saves another 16%.

RL objective:

Learning from stale samples without drifting

Core lesson 5: Correct stale, low-precision samples rather than discarding them, and treat rising entropy as an early warning.

Rollouts are sampled at FP8 precision on weights lagging up to 6 versions behind, slightly different from the BF16 policy being trained. Rather than discarding these stale trajectories, the team corrects for them directly in the training objective. The core algorithm is GRPO, with 12 attempts per task and one optimizer update per step; the advantage of each attempt includes both a reward term and a length reward term:

A = (R′ − mean R′) / std R′ + (b − mean b) / 0.5

(mean and standard deviation are computed over the 12 attempts of the same task)

R′ = 1.5 (when the attempt solves the task); otherwise R′ = 0.5 × R (R is the final reward in [0, 1] described above)

b = 0.2 × (1 − tokens / the maximum token count among solved attempts in the group), computed for solved attempts when at least 2 attempts in the group solve the task, otherwise 0.

Truncated importance sampling (TIS). Each token is weighted by the ratio of the BF16 trainer probability to the FP8 rollout probability, with the ratio clipped to [0.5, 2.0] to limit the influence of any single token; only 1%–3% of tokens hit the clipping boundary.

Score centering (SC). The rollout engine returns top-32 log probabilities for each token, and the trainer subtracts the sampler's expected score over these candidates from each token's policy gradient score. This eliminates the residual drift toward the stale FP8 sampler that clipping alone cannot remove, at the cost of a 7.6% increase in per-step time.

Success offset. A full solve (R = 1.0) counts as 1.5 in the advantage, while unsolved attempts count as half of their partial score: the best "near miss" (R = 0.5) counts as only 0.25, whereas a solve counts as 1.5. Partial scores can shape learning but are never equated with a genuine solve.

Loss normalization floor. The loss of each attempt is normalized by its token count, with a denominator of at least 2,048 tokens, preventing degenerate rollouts with only a handful of tokens from receiving an excessively large per-token weight and dominating the gradient.

With these corrections, training uses neither a KL penalty nor an entropy bonus. In healthy training, policy entropy slowly decreases from about 0.46 to 0.33; a rise in entropy is an early warning signal of training instability.

Experiments: across-the-board gains on three benchmarks,

standing on the cost–performance frontier

FireFlash is evaluated in the same harness used during training and compared against the official results of its starting point GLM-5.3-Flash and the public results of other models (Figure 5). FireFlash surpasses the base model on all three benchmarks: Terminal-Bench 4.0 improves by 10.4 points, exceeding Fable 5 and the full-size sibling model GLM-5.3; DeepSWE v1.1 improves by 5.4 points, exceeding Fable 5.1; SWE-Bench Pro Verified improves by 5.1 points, exceeding Kimi K3 and GPT-5.6 Sol.

Figure 5: Pass@1 on three benchmarks. Orange is FireFlash, blue is the official result of its starting point GLM-5.3-Flash, and the rest are the public results of the respective models.

Cost and performance

On Terminal-Bench 4.0, FireFlash lies on the cost–performance Pareto frontier (Figure 1): among the more than forty models and reasoning-effort settings in the figure, only 4 have a lower per-trial cost than it, and none of them scores above 35.8%. At GLM-5.3-Flash's public pricing, FireFlash costs $0.72 per trial, compared with $22.02 for Fable 5 at maximum reasoning effort; the next point on the frontier, GPT-6.1 Terra, scores 55.6% at a cost of $1.49. Every model that scores higher than FireFlash costs at least twice as much per trial; the full-size sibling model GLM-5.3 costs $8.27 per trial at maximum reasoning effort yet scores only 41.8%.

Conclusion

FireFlash shows that targeted post-training can turn an open-weight model into a highly capable domain-specific agent, reaching frontier-level performance at a fraction of the inference cost. The authors argue that jointly optimizing the model, training environment, harness, and inference system is an alternative path to building powerful and efficient agents beyond simply scaling up. FireFlash is a first step in this direction, and the team will next extend this approach from coding and terminal tasks to more complex, longer-horizon work that requires repeated execution.

About the authors

Wenbo Hu is a PhD student in Computer Science at UCLA, advised by Professors Kai-Wei Chang and Nanyun Peng. His research focuses on agentic reinforcement learning (Agentic RL), with interests in large language models, multimodality, and agents; his representative works include Tev, MQT-LLaVA, and 3DLLM-Mem.

The other authors of this article are Jesse Cai, Po-nien Kung, Yuxuan Lu, Di Wu, and Huimin Zeng (in alphabetical order by surname), as well as Professors Nanyun Peng, Dakuo Wang, and Kai-Wei Chang.

© THE END

For reprints, please contact this official account for authorization.

Submissions or story tips: [email protected]

Open in WeChat