Termsslug: swe-bench

SWE-bench

Benchmark evaluating models and agents on real software engineering issues from GitHub repositories.

Curated Intelligence

11 items · Page 1/2
Rohan Paul (@rohanpaul_ai)T2·Agents
78 pts
Berkeley and Microsoft Cut Coding Agent Token Costs by Up to 30.9%

UC Berkeley and Microsoft introduce TokenEconomizer, an external black-box controller that learns from past runs which facts to inject, which unused tools to prune, and when to halt failing loops, changing only prompts and tool settings. Across Claude Code, Codex, and GitHub Copilot CLI on 277 SWE-Bench tasks, it cut mean per-task cost by 18.4% to 30.9% and improved Pass@1 in 4 of 5 agent-model settings.

Rohan Paul (@rohanpaul_ai)T2·Tools
62 pts
ByteAsk Open-Sources C/C++ Coding Agent Cutting Fix Costs by 68%

ByteAsk launched an open-source C/C++ coding agent whose upfront context engine locates the fix site before the model reads code, cutting tokens per run from 1,795K to 395K (about 78%) and cost by 68.1% while fixing the same number of bugs. It also flags sibling sites needing the same edit and validates patches only against covering tests, saving roughly 80% on obscure projects and hard issues.

Marktechpost (@Marktechpost)T2·Tools
78 pts
Microsoft Open-Sources TokenEconomizer to Cut Coding Agent Costs

Microsoft Research and UC Berkeley have open-sourced TokenEconomizer, a plugin designed to reduce the operational costs of coding agents. By learning repository-specific facts from past sessions, pruning unused tool schemas, and redirecting looping failures, it optimizes context usage. Benchmarks on SWE-Bench show an average cost reduction of 18.4% to 30.9%, albeit with a slight dip in Pass@1 accuracy.

Hacker News AIT2·Tools
68 pts
Open-Source Benchmark Parsimony Quantifies AI Code Bloat Using Net Syntax Units Instead of Line Counts

Parsimony is an open-source benchmark addressing the neglect of code bloat in existing evaluations. By counting 'net coding units' (syntax elements like statements and calls) rather than lines of code, and subtracting deleted portions, it provides a more accurate measure of AI-generated code conciseness and maintainability. Leveraging SWE-bench Verified data, it offers developers a new dimension for assessing LLM code efficiency.

量子位官网T2·Agents
48 pts
Lenovo's TianxiCode Agent Tops SWE-bench-Live with 71% Resolution Rate

Lenovo's self-developed code agent framework TianxiCode, paired with DeepSeek-v4.1-Flash, achieved a global top ranking on SWE-bench-Live with a 71% issue resolution rate and official Verified certification. This result highlights the critical role of specialized agent architectures in unlocking model potential for complex software engineering tasks, with plans to integrate into Lenovo's AI hardware products.

机器之心 (微信公众号)T2·Agents
86 pts
LegoFlow Open-Sourced: Agents Automate Full Code Data Pipeline and Training

Huawei and HKU researchers released LegoFlow, an open-source framework that automates the entire code data pipeline from repository discovery to training and evaluation. The accompanying LegoFlow-SWE dataset features 5,000 high-quality tasks; fine-tuning Qwen3.5-35B-A3B-Base on just 1,000 trajectories achieves 70.2% on SWE-bench Verified. The system also demonstrates recursive self-improvement, boosting model performance from 7.6% to 64.4% over two iterations.

Kimi 博客 (月之暗面)T1·Models
86 pts
Moonshot AI Open-Sources Kimi K2.6 for Long-Horizon Agents

Moonshot AI has open-sourced Kimi K2.6, emphasizing state-of-the-art coding, long-horizon execution, and agent swarm capabilities. Benchmark data shows it outperforms GPT-5.4 and Claude Opus 4.6 in key coding metrics like SWE-Bench Pro, offering a highly competitive baseline despite trailing the internal flagship K3.

智谱 AI 研究T1·Models
85 pts
Zhipu AI Open-Sources GLM-5.1 with 8-Hour Autonomous Coding

Zhipu AI has released and open-sourced its flagship model, GLM-5.1, marking a breakthrough in long-horizon tasks by enabling over 8 hours of autonomous coding to deliver complete engineering solutions. The model sets new global records on key benchmarks like SWE-Bench Pro, ranking third overall and first among open-source models.

OpenAI 官方博客T1·Models
45 pts
OpenAI Releases GPT-5 Practical Guide: Emphasizes Prompt Refinement and Agentic Control with 74.9% SWE-bench Score

OpenAI has released a practical developer guide for the GPT-5 model series, highlighting its enhanced capabilities in coding and agentic tasks alongside greater steerability. The guide emphasizes that prompts optimized for previous models like GPT-4.1 do not translate directly to GPT-5, requiring developers to refine their prompting strategies to align with the new model's unique behavioral patterns. It reveals that GPT-5 achieves a 74.9% score on SWE-bench Verified, marking a significant leap in complex multi-step reasoning and workflow orchestration.

OpenAI 官方博客T1·Models
92 pts
OpenAI Launches GPT-5: SOTA Coding & Agentic Performance

OpenAI has officially released GPT-5 on its API platform, positioning it as the best model for coding and agentic tasks. It achieves state-of-the-art performance with 74.9% on SWE-bench Verified and excels in frontend development and complex codebase maintenance. Early testers report significantly lower tool-calling error rates and improved hallucination control compared to previous frontier models.