Agent Arena Data Reveals 66% Cost Gap in Long-Horizon Tasks
Arena.ai released new benchmark data from Agent Arena showing significant cost disparities among frontier models in complex tasks lasting over an hour. Gemini 4 Carbon emerged as the most cost-efficient option with a median hourly cost of $10.51, undercutting Claude Opus 5.5 by 50% and GPT-6 Astra by 66%. While Claude Opus 5.5 leads in capability scores, its high time and monetary costs force developers to re-evaluate the performance-budget trade-off for long-horizon agent workflows.
Full Report
Arena.ai (@arena) · 2026-11-04 17
In the last 14 days, nearly 30,000 tasks ran for over an hour on Agent Arena. Within that hour, the average model made 332 tool calls.
In these long-horizon tasks, the right model choice can cut the cost by up to 66%.
In Agent Arena, an hour of work costs a median $10.51 per task with Gemini 4 Carbon, $21.13 with Claude Opus 5.5 by @AnthropicAI and $31.08 with GPT-6 Astra by @OpenAI.
That makes Gemini 4 Carbon by @GoogleDeepmind 50% cheaper than Opus and 66% cheaper than Astra for an hour-long task.
For long-running jobs, weighing cost against capability becomes critical: Claude Opus 5.5 still scores higher, but costs twice as much per hour of work.
↳ 引用 @arena (Arena.ai): Agent Arena measures complex, real-world tasks. Data shows that these long-horizon sessions amplify differences in frontier models.
Long-horizon tasks in Agent Arena span hundreds of turns, from porting a codebase to Rust and auditing an investment strategy document to writing a math textbook, and more.
Comparing the top model from each of the 5 leading labs on Agent Arena reveals a 55-minute gap in how long they spend on long-horizon tasks. The models are:
- Claude Opus 5.5 by @AnthropicAI
- Gemini 4 Carbon by @GoogleDeepMind
- GPT-6 Astra by @OpenAI
- Muse Wonder Full by @AIatMeta
- Kimi K3.1 Pro by @Kimi_Moonshot
In this group, Claude Opus 5.5, #1 in Agent Arena, takes the longest at every task length, peaking at 1 hour 51 minutes. In contrast, Kimi K3.1 Pro spends the least time at every length, with 56 minutes on long-horizon tasks.
