HOT RADAR
agentHEAT 12.0°

AI CURATED HOT STORY · 11/17/2026

Coding Agent Benchmark Finds 64% of Inference Spend Wasted on Failures

1 reports archived1 independent sourcesupdated 11/17/2026, 22:53:43
Synthesis & Latest Updates
1 Sources Cross-Validated

Runta and Baseten released the FrontierHarness v1.1 leaderboard, spanning 22 agent harnesses, 5 models and 62 tasks across 27,280 runs and 83B tokens. The headline finding: switching models should come with switching harnesses, since each model's best value-for-money harness differs and cost rankings shift by a median of 11 of 22 places, with Codex CLI falling from 1st with GLM-5.3 to 16th with Kimi-K3. Notably, 64% of inference spend went to failed runs, and neither tool count nor cache hit rate predicted cost per pass, only step count did.

LATEST/Runta and Baseten released the FrontierHarness v1.1 leaderboard, spanning 22 agent harnesses, 5 models and 62 tasks across 27,280 runs and 83B tokens. The headline finding: switching models should come with switching harnesses, since each model's best value-for-money harness differs and cost rankings shift by a median of 11 of 22 places, with Codex CLI falling from 1st with GLM-5.3 to 16th with Kimi-K3. Notably, 64% of inference spend went to failed runs, and neither tool count nor cache hit rate predicted cost per pass, only step count did.

Hourly heat curve

8 fully observed hours
Now
12.0
Peak
14.311-18
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first