返回全部资讯
OpenAI 官方博客T1·基座大模型

OpenAI 发布 GPT-5:SWE-bench 达 74.9%,主打编码与智能体协作

原标题:Introducing GPT-5 for developers
92AI 研判分

OpenAI 正式在 API 平台推出 GPT-5,宣称其为迄今最强的编码与智能体任务模型。该模型在 SWE-bench Verified 上取得 74.9% 的 SOTA 成绩,并在前端开发及复杂代码库修复中表现卓越,具备高可控性与工具调用前的解释能力。早期测试显示其显著降低了工具调用错误率并提升了幻觉抑制能力。

报道正文

本文目录18 个章节

简介

今天,我们在 API 平台发布了 GPT‑5——这是我们迄今为止在编程和智能体(agentic)任务上表现最佳的模型。

GPT‑5 在关键编程基准测试中达到了最先进水平(SOTA),在 SWE-bench Verified 上得分 74.9%,在 Aider polyglot 上得分 88%。我们将 GPT‑5 训练为真正的编程协作者。它在生成高质量代码以及处理修复 Bug、编辑代码和回答复杂代码库相关问题等任务方面表现出色。该模型具有可引导性和协作性——能够高精度地遵循非常详细的指令,并能在工具调用之前及之间对其操作提供预先解释。此外,该模型在前端编程方面也表现出色,在内部测试中,其在 Web 前端开发任务上有 70% 的情况优于 OpenAI o3。

我们与来自初创企业和大型企业的早期测试者合作,基于真实世界的编程任务对 GPT‑5 进行了训练。Cursor 表示 GPT‑5 是“[他们]使用过的最聪明的模型”,并且“极其智能、易于引导,甚至具备[他们在其他模型中]未曾见过的个性”。Windsurf 分享称 GPT‑5 在其评估集上达到 SOTA,且“与其他前沿模型相比,其工具调用错误率减半”。Vercel 表示“它是最好的前端 AI 模型,在审美感知和代码质量两方面均达到顶尖性能,使其自成一类。”

GPT‑5 在长时间运行的智能体任务中也表现出色——在仅发布 2 个月的工具调用基准测试 τ 2-bench telecom 上取得了 SOTA 结果(96.7%)。GPT‑5 改进的工具智能使其能够可靠地串联数十次工具调用——无论是顺序执行还是并行执行——而不会迷失方向,从而极大地提升了端到端执行复杂现实世界任务的能力。它还能更精确地遵循工具指令,更好地处理工具错误,并在长上下文内容检索方面表现出色。Manus 表示 GPT‑5 “在[他们的]内部基准测试中实现了[他们]从未在任何单一模型上看到过的最佳性能”。Notion 表示“[该模型]的快速响应,尤其是在低推理模式下,使 GPT‑5 成为需要一次性解决复杂任务时的理想模型”。Inditex 分享道,“真正让 [GPT‑5] 脱颖而出的是其推理的深度:细致入微、多层次的答案反映了其对主题领域的真正理解。”

我们正为 API 引入新功能,赋予开发者对模型响应更强的控制力。GPT‑5 支持新的 verbosity 参数(取值:low、medium、high),帮助控制回答是简洁明了还是详尽全面。此外,GPT‑5 的 reasoning_effort 参数现在可设为 minimal,从而在不进行大量推理的情况下更快获得答案。我们还新增了一种工具类型——自定义工具(custom tools),允许 GPT‑5 使用纯文本而非 JSON 调用工具。自定义工具支持通过开发者提供的上下文无关文法(context-free grammars)进行约束。

我们在 API 中发布三种尺寸的 GPT‑5 模型——gpt-5、gpt-5-mini 和 gpt-5-nano,以便开发者在性能、成本和延迟之间更灵活地权衡取舍。虽然 ChatGPT 中的 GPT‑5 是由推理模型、非推理模型和路由模型组成的系统,但 API 平台中的 GPT‑5 是为 ChatGPT 提供最大性能支持的推理模型。值得注意的是,采用最小推理模式的 GPT‑5 与 ChatGPT 中的非推理模型是不同的模型,且针对开发者进行了更好的调优。ChatGPT 中使用的非推理模型可通过 gpt-5-chat-latest 获取。

如需了解 ChatGPT 中的 GPT‑5 及其他 ChatGPT 改进详情,请参阅我们的研究博客。关于企业用户对使用 GPT‑5 的热情,请参见我们的企业博客⁠。

编程

GPT‑5 是我们迄今发布的编程能力最强的模型。它在各类编程基准测试和实际应用场景中均优于 o3,并经过专门微调,以在 Cursor、Windsurf、GitHub Copilot 和 Codex CLI 等智能体编程产品中表现出色。GPT‑5 给内测用户留下了深刻印象,在许多内部私有评估中创下纪录。

GPT‑5 在实际编程任务中的早期反馈

在基于真实软件工程任务的 SWE-bench Verified 评估中,GPT‑5 得分 74.9%,高于 o3 的 69.1%。值得注意的是,GPT‑5 以更高效的方式实现了高分:相较于高推理努力下的 o3,GPT‑5 的输出 token 数量减少了 22%,工具调用次数减少了 45%。

在 SWE-bench Verified⁠ 中,模型会收到一个代码仓库和问题描述,并必须生成补丁以解决该问题。文本标签表示推理努力程度。我们的评分排除了 500 个问题中的 23 个,因为它们的解决方案在我们的基础设施上无法可靠通过。GPT‑5 使用了一个强调彻底验证解决方案的简短提示;同样的提示对 o3 没有带来益处。

在 Aider polyglot(一项代码编辑评估)中,GPT‑5 创下了 88% 的新纪录,与 o3 相比错误率降低了三分之一。

在 Aider polygot⁠(opens in a new window) (diff) 中,模型会收到来自 Exercism 的一个编程练习,并必须以代码 diff 的形式写出其解决方案。推理模型在高推理努力度下运行。

我们还发现 GPT‑5 在深入挖掘代码库以回答关于各个部分如何工作或互操作的问题方面表现出色。在像 OpenAI 的强化学习栈这样复杂的代码库中,我们发现 GPT‑5 可以帮助我们推理并回答关于我们代码的问题,从而加速我们自己的日常工作。

前端工程

在为 Web 应用生成前端代码时,GPT‑5 更具审美意识、更雄心勃勃且更准确。在与 o3 的并列比较中,70% 的情况下测试者更偏好 GPT‑5。

以下是一些精选示例,展示 GPT‑5 仅凭单个提示就能完成的任务:

在此处⁠(opens in a new window) 查看 GPT‑5 的更多示例。

编码协作

GPT‑5 是一个更好的协作者,特别是在 Cursor、Windsurf、GitHub Copilot 和 Codex CLI 等 agentic coding 产品中。在工作过程中,GPT‑5 可以在工具调用之间输出计划、更新和总结。与我们过去的模型相比,GPT‑5 在完成雄心勃勃的任务时更加主动,无需暂停等待你的许可,也不会因高复杂度而退缩。

以下是 GPT‑5 在处理复杂任务时的表现示例(在本例中是为一家餐厅创建网站):

在用户请求为其餐厅创建一个网站后,GPT‑5 会分享一个快速计划,搭建应用脚手架,安装依赖项,创建站点内容,运行构建以检查编译错误,总结其工作成果,并建议潜在的后续步骤。此视频已加速约 3 倍以节省您的等待时间;创建该网站的完整耗时约为三分钟。

Agentic 任务

除了 agentic coding(智能体编程)之外,GPT‑5 在一般的 agentic 任务上也表现更佳。GPT‑5 在指令遵循基准测试(Scale MultiChallenge 上得分 69.6%,由 o3‑mini 评分)和工具调用基准测试(τ 2-bench telecom 上得分 96.7%)中均创下新纪录。改进的工具智能使 GPT‑5 能够更可靠地串联多个动作,以完成现实世界中的任务。

GPT‑5 在 agentic 任务上的早期反馈

指令遵循

GPT‑5 比其任何前身都更可靠地遵循指令,在 COLLIE、Scale MultiChallenge 以及我们内部的指令遵循评估中均获得高分。

在 COLLIE⁠(在新窗口中打开) 中,模型必须编写满足各种约束条件的文本。在 Scale MultiChallenge⁠(在新窗口中打开) 中,模型在多轮对话中被挑战,要求正确利用来自先前消息的四类信息。我们的分数是使用 o3‑mini 作为评分器得出的,其准确性高于 GPT‑4o。在我们内部的 OpenAI API 指令遵循评估中,模型必须遵循源自真实开发者反馈的高难度指令。推理模型均以高推理强度运行。

工具调用

我们努力在开发者关心的方面改进工具调用能力。GPT‑5 更好地遵循工具指令,更好地处理工具错误,并且更擅长按顺序或并行主动发起多次工具调用。在收到指示时,GPT‑5 还可以在工具调用之前和之间输出前言消息,以便在较长的 agentic 任务期间向用户更新进度。

两个月前,Sierra.ai 发布了 τ 2-bench telecom,这是一个具有挑战性的工具使用基准测试,突出了当语言模型与可由用户改变的环境状态交互时,其性能会显著下降的情况。在他们的 出版物⁠(在新窗口中打开) 中,没有任何模型的得分超过 49%。而 GPT‑5 的得分为 97%。

[注:本文篇幅超长,以上为核心前篇精译,后续内容保留原文呈现]

In τ2-bench⁠(opens in a new window), a model must use tools to accomplish a customer service task, where there may be a user who can communicate and can take actions on the world state. Reasoning models were run with high reasoning effort.

GPT‑5 shows strong improvements to long-context performance as well. On OpenAI-MRCR, a measure of long-context information retrieval, GPT‑5 outperforms o3 and GPT‑4.1, by a margin that grows substantially at longer input lengths.

In OpenAI-MRCR⁠(opens in a new window)(multi-round co-reference resolution), multiple identical “needle” user requests are inserted into long “haystacks” of similar requests and responses, and the model is asked to reproduce the response to i-th needle. Mean match ratio measures the average string match ratio between the model’s response and the correct answer. The points at 256k max input tokens represent averages over 128k–256k input tokens, and so forth. Here, 256k represents 256 * 1,024 = 262,114 tokens. Reasoning models were run with high reasoning effort.

We’re also open sourcing BrowseComp Long Context⁠(opens in a new window), a new benchmark for evaluating long-context Q&A. In this benchmark, the model is given a user query, a long list of relevant search results, and must answer the question based on the search results. We designed BrowseComp Long Context to be realistic, difficult, and have reliably correct ground truth answers. On inputs that are 128K–256K tokens, GPT‑5 gives the correct answer 89% of the time.

In the API, all GPT‑5 models can accept a maximum of 272,000 input tokens and emit a maximum of 128,000 reasoning & output tokens, for a total context length of 400,000 tokens.

Factuality

GPT‑5 is more trustworthy than our prior models. On prompts from LongFact and FactScore benchmarks, GPT‑5 makes ~80% fewer factual errors than o3. This makes it better suited for agentic use cases where correctness matters—especially in code, data, and decision-making.

Higher scores are worse. LongFact⁠(opens in a new window) and FActScore⁠(opens in a new window) consist of open-ended fact-seeking questions. We use an LLM-based grader with browsing to fact-check responses on prompts from these benchmarks and measure the fraction of factually incorrect claims. Implementation and grading details can be found in the system card⁠. Reasoning models used high reasoning effort. Search was not enabled.

Generally, GPT‑5 has been trained to be more self-aware of its own limitations and better able to handle unexpected curveballs. We also trained GPT‑5 to be much more accurate on health questions (read more in our research blog). As with all language models, we recommend you verify GPT‑5’s work when the stakes are high.

New features

Minimal reasoning effort

Developers can control GPT‑5’s thinking time via the reasoning_effort parameter in the API. In addition to the prior values—low, medium (default), and high—GPT‑5 also supports minimal, which minimizes GPT‑5’s reasoning to return an answer quickly.

Higher reasoning_effort values maximize quality and lower values maximize speed. Not all tasks benefit equally from additional reasoning, so we recommend experimenting to see which works best for the use cases you care about.

For example, reasoning above low adds little to relatively simple long-context retrieval, but adds quite a few percentage points to CharXiv Reasoning⁠(opens in a new window), a visual reasoning benchmark.

GPT‑5’s reasoning effort yields different benefits on different tasks. For CharXiv Reasoning, GPT‑5 was given access to a python tool.

Verbosity

To help steer the default length of GPT‑5’s answers, we’ve introduced a new API parameter verbosity, which takes values of low, medium (default), and high. If explicit instructions conflict with the verbosity parameters, explicit instructions take precedent. For example, if you ask GPT‑5 to “write a 5 paragraph essay”, the model’s response should always be 5 paragraphs regardless of the verbosity level (however, the paragraphs themselves may be longer or shorter).

Preamble messages before tool calls

If instructed, GPT‑5 will output user-visible preamble messages before and between tool calls. Unlike hidden reasoning messages, these visible messages allow GPT‑5 to communicate plans and progress to the user, helping end users understand its approach and intent behind the tool calls.

Custom tools

We’re introducing a new tool type—custom tools—that allows GPT‑5 to call a tool with plaintext instead of JSON. To constrain GPT‑5 to follow custom tool formats, developers can supply a regex, or even a more fully specified context-free grammar⁠(opens in a new window).

Previously, our interface for developer-defined tools required them to be called with JSON, a common format used by web APIs and developers generally. However, outputting valid JSON requires the model to perfectly escape all quotation marks, backslashes, newlines, and other control characters. Although our models are well-trained to output JSON, on long inputs like hundreds of lines of code or a 5-page report, the odds of an error creep up. With custom tools, GPT‑5 can write tool inputs as plaintext, without having to escape all of the characters that require escaping.

On SWE-bench Verified using custom tools instead of JSON tools, GPT‑5 scores about the same.

Safety

GPT‑5 advances the frontier on safety and is a more robust, reliable, and helpful model. GPT‑5 is significantly less likely to hallucinate than our previous models, more honestly communicates its actions and capabilities to the user and provides the most helpful answer where possible while still staying within safety boundaries. You can read more in our research blog.

Availability & pricing

GPT‑5 is available now in the API platform in three sizes: gpt-5, gpt-5-mini, and gpt-5-nano. It’s available on the Responses API, Chat Completions API, and is the default in Codex CLI. GPT‑5 is priced at 1.25/1Minputtokensand1.25/1M input tokens and 10/1M output tokens, GPT‑5 mini is priced at 0.25/1Minputtokensand0.25/1M input tokens and 2/1M output tokens, and GPT‑5 nano is priced at 0.05/1Minputtokensand0.05/1M input tokens and 0.40/1M output tokens.

These models support the reasoning_effort and verbosity API parameters, as well as custom tools. They also support parallel tool calling, built-in tools (web search, file search, image generation, and more), core API features (streaming, Structured Outputs, and more), and cost-saving features such as prompt caching and Batch API.

The non-reasoning version of GPT‑5 used in ChatGPT is available in the API as gpt-5-chat-latest, also priced at 1.25/1Minputtokensand1.25/1M input tokens and 10/1M output tokens.

GPT‑5 is also launching across Microsoft platforms, including Microsoft 365 Copilot, Copilot, GitHub Copilot, and Azure AI Foundry.

Detailed benchmarks

Intelligence
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
AIME ’25(no tools)94.6%91.1%85.2%88.9%92.7%46.4%40.2%-
FrontierMath(with python tool only)26.3%22.1%9.6%15.8%15.4%---
GPQA diamond(no tools)85.7%82.3%71.2%83.3%81.4%66.3%65.0%50.3%
HLE1tools24.8%16.7%8.7%20.2%14.7%5.4%3.7%-
HMMT 2025(no tools)93.3%87.8%75.6%81.7%85.0%28.9%35.0%-

[1] There is a small discrepancy with numbers reported in our previous blog post, as those were run on a former version of HLE.

Multimodal
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
MMMU84.2%81.6%75.6%82.9%81.6%74.8%72.7%55.4%
MMMU-Pro(avg across standard and vision sets)78.4%74.1%62.6%76.4%73.4%60.3%58.9%33.0%
CharXiv reasoning(python enabled)81.1%75.5%62.7%78.6%72.0%56.7%56.8%40.5%
VideoMMMU, max frame 25684.6%82.5%66.8%83.3%79.4%60.9%55.1%30.2%
ERQA65.7%62.9%50.1%64.0%56.5%44.3%42.3%26.5%
Coding
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
SWE-Lancer: IC SWE Diamond Freelance Coding Tasks$112K$75K$49K$86K$66K$34K$31K$9K
SWE-bench Verified[2]74.9%71.0%54.7%69.1%68.1%54.6%23.6%-
Aider polyglot(diff)88.0%71.6%48.4%79.6%58.2%52.9%31.6%6.2%

[2] We omit 23/500 problems that could not run on our infrastructure. The full list of 23 tasks omitted are 'astropy__astropy-7606', 'astropy__astropy-8707', 'astropy__astropy-8872', 'django__django-10097', 'django__django-7530', 'matplotlib__matplotlib-20488', 'matplotlib__matplotlib-20676', 'matplotlib__matplotlib-20826', 'matplotlib__matplotlib-23299', 'matplotlib__matplotlib-24970', 'matplotlib__matplotlib-25479', 'matplotlib__matplotlib-26342', 'psf__requests-6028', 'pylint-dev__pylint-6528', 'pylint-dev__pylint-7080', 'pylint-dev__pylint-7277', 'pytest-dev__pytest-5262', 'pytest-dev__pytest-7521', 'scikit-learn__scikit-learn-12973', 'sphinx-doc__sphinx-10466', 'sphinx-doc__sphinx-7462', 'sphinx-doc__sphinx-8265', and 'sphinx-doc__sphinx-9367'.

Instruction Following
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
Scale multichallenge3grader69.6%62.3%54.9%60.4%57.5%46.2%42.2%31.1%
Internal API instruction following eval(hard)64.0%65.8%56.1%47.4%44.7%49.1%45.1%31.6%
COLLIE99.0%98.5%96.9%98.4%96.1%65.8%54.6%42.5%

[3] Note: we find that the default grader in MultiChallenge (GPT-4o) frequently mis-scores model responses. We find that swapping the grader to a reasoning model, like o3-mini, improves accuracy on grading significantly on samples we’ve inspected.

Function Calling
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
Tau 2-bench airline62.6%60.0%41.0%64.8%60.2%56.0%51.0%14.0%
Tau 2-bench retail81.1%78.3%62.3%80.2%70.5%74.0%66.0%21.5%
Tau 2-bench telecom96.7%74.1%35.5%58.2%40.5%34.0%44.0%12.1%
Long Context
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
OpenAI-MRCR: 2 needle 128k95.2%84.3%43.2%55.0%56.4%57.2%47.2%36.6%
OpenAI-MRCR: 2 needle 256k86.8%58.8%34.9%--56.2%45.5%22.6%
Graphwalks bfs <128k78.3%73.4%64.0%77.3%62.3%61.7%61.7%25.0%
Graphwalks parents <128k73.3%64.3%43.8%72.9%51.1%58.0%60.5%9.4%
BrowseComp Long Context 128k90.0%89.4%80.4%88.3%80.0%85.9%89.0%89.4%
BrowseComp Long Context 256k88.8%86.0%68.4%--75.5%81.6%19.1%
VideoMME(long, with subtitle category)86.7%78.5%65.7%84.9%79.5%78.7%68.4%55.2%
Hallucinations
GPT-5(high)GPT-5 mini(high)GPT-5 nano(high)OpenAI o3(high)OpenAI o4-mini(high)GPT-4.1GPT-4.1 miniGPT-4.1 nano
LongFact-Concepts hallucination rate(no tools)[lower is better]1.0%0.7%1.0%5.2%3.0%0.7%1.1%-
LongFact-Objects hallucination rate(no tools)[lower is better]1.2%1.3%2.8%6.8%8.9%1.1%1.8%-
FActScore hallucination rate(no tools)[lower is better]2.8%3.5%7.3%23.5%38.7%6.7%10.9%-