OpenAI 详解 Codex CLI 智能体循环架构

原标题:Unrolling the Codex agent loop
68AI 研判分

OpenAI 发布技术博客,深入解析其本地软件智能体 Codex CLI 的核心“智能体循环”(Agent Loop)机制。文章阐述了该循环如何协调用户、模型与工具调用以执行高质量代码变更,并预告后续将探讨沙箱隔离与架构细节。

报道正文

本文目录6 个章节

Codex CLI⁠(在新窗口中打开) 是我们的跨平台本地软件代理,旨在安全、高效地在你的机器上运行,同时生成高质量、可靠的软件变更。自我们于四月首次发布 CLI 以来⁠,我们在如何构建世界一流的软件代理方面积累了大量经验。为了分享这些洞察,这是我们将持续推出的系列文章中的第一篇,我们将探讨 Codex 工作原理的各个方面以及来之不易的经验教训。(如需更细致地了解 Codex CLI 的构建方式,请查看我们的开源仓库 https://github.com/openai/codex⁠(在新窗口中打开)。如果你希望了解更多细节,我们在 GitHub issues 和 pull requests 中记录了许多设计决策的细节。)

作为开篇,我们将聚焦于 agent loop(代理循环),这是 Codex CLI 的核心逻辑,负责协调用户、模型以及模型为执行有意义的软件工作而调用的工具之间的交互。我们希望这篇文章能让你清楚地了解我们的代理(或称“harness”,即执行框架)在利用 LLM 时所扮演的角色。

在深入探讨之前,先简要说明一下术语:在 OpenAI,“Codex”涵盖了一系列软件代理产品,包括 Codex CLI、Codex Cloud 和 Codex VS Code 扩展。本文重点介绍 Codex harness,它提供了支撑所有 Codex 体验的核心 agent loop 和执行逻辑,并通过 Codex CLI 呈现出来。为了方便起见,下文将互换使用“Codex”和“Codex CLI”这两个术语。

Agent Loop

每个 AI 代理的核心都是所谓的“agent loop”。简化的 agent loop 示意图如下所示:

首先,代理接收来自用户的 input(输入),并将其包含在准备给模型的一组文本指令中,这组指令被称为 prompt(提示词)。

下一步是查询模型,发送我们的指令并要求其生成响应,这一过程称为 inference(推理)。在推理过程中,文本 prompt 首先被转换为一串输入 tokens⁠(在新窗口中打开)——即用于索引模型词汇表的整数。随后,这些 tokens 被用于对模型进行采样,从而生成新的输出 token 序列。

输出 token 会被转换回文本,从而形成模型的响应。由于 token 是逐步生成的,这种转换可以在模型运行过程中实时发生,这也是许多基于 LLM 的应用程序支持流式输出的原因。在实际应用中,推理过程通常封装在基于文本的 API 之后,从而隐藏了分词(tokenization)的细节。

作为推理步骤的结果,模型要么 (1) 针对用户的原始输入生成最终响应,要么 (2) 请求一个代理(agent)需要执行的 工具调用(tool call)(例如,“运行 ls 并报告输出”)。在情况 (2) 中,代理执行该工具调用,并将其输出追加到原始提示词(prompt)中。此输出用于生成新的输入以重新查询模型;随后,代理可以利用这些新信息再次尝试。

这一过程会不断重复,直到模型停止发出工具调用,转而生成一条面向用户的消息(在 OpenAI 模型中称为 assistant message)。在许多情况下,这条消息直接回答了用户的原始请求,但它也可能是一个向用户提出的后续问题。

由于代理可以执行修改本地环境的工具调用,其“输出”并不局限于 assistant message。在许多情况下,软件代理的主要输出是在你的机器上编写或编辑的代码。尽管如此,每一轮对话总是以一条 assistant message 结束——例如“我已添加了你要求的 architecture.md”——这标志着代理循环中的终止状态。从代理的角度来看,其工作已完成,控制权交还给用户。

图中所示的从 user input 到 agent response 的过程被称为一次对话 turn(在 Codex 中称为 thread)。尽管这个 conversation turn 可能包含 model inference 和 tool calls 之间的多次迭代。每次你向现有对话发送新消息时,对话历史都会作为新 turn 提示词的一部分被包含在内,其中包括之前各 turn 的消息和工具调用:

这意味着随着对话的增长,用于采样模型的 prompt 长度也会增加。这个长度至关重要,因为每个模型都有一个 context window(上下文窗口),即单次推理调用中可使用的最大 token 数量。请注意,该窗口同时包含输入 和 输出 token。可以想象,agent 可能会在单个回合中决定发起数百次工具调用,从而耗尽上下文窗口。因此,上下文窗口管理 是 agent 众多职责之一。现在,让我们深入探讨 Codex 如何运行 agent loop。

模型推理

Codex CLI 向 Responses API⁠(opens in a new window) 发送 HTTP 请求以执行模型推理。我们将研究信息如何在 Codex 中流动,Codex 使用 Responses API 来驱动 agent loop。

Codex CLI 使用的 Responses API 端点是可配置的⁠(opens in a new window),因此它可以与任何实现了 Responses API⁠(opens in a new window)的端点配合使用:

让我们探索 Codex 如何为对话中的首次推理调用创建 prompt。

构建初始 prompt

作为终端用户,在查询 Responses API 时,你无需逐字指定用于采样模型的提示词(prompt)。相反,你需要在查询中指定各种输入类型,由 Responses API 服务器决定如何将这些信息组织成模型设计所消费的提示词结构。你可以将提示词视为一个“条目列表”;本节将解释你的查询是如何被转换为该列表的。

在初始提示词中,列表中的每个条目都与一个角色(role)相关联。role 指示了关联内容应具有的权重,其取值按优先级从高到低依次为:system、developer、user、assistant。

Responses API⁠(opens in a new window) 接收包含许多参数的 JSON 载荷。我们将重点关注以下三个参数:

在 Codex 中,如果指定了 ~/.codex/config.toml 中的 model_instructions_file⁠(opens in a new window),则从该文件读取 instructions 字段;否则,使用与模型关联的 基础指令(base instructions)⁠(opens in a new window)。特定于模型的指令位于 Codex 仓库中,并打包进 CLI(例如 gpt-5.2-codex_prompt.md⁠(opens in a new window))。

tools 字段是一个符合 Responses API 定义模式的工具定义列表。对于 Codex,这包括由 Codex CLI 提供的工具、应由 Responses API 提供并供 Codex 使用的工具,以及通常通过 MCP 服务器由用户提供的工具:

[注:本文篇幅超长,以上为核心前篇精译,后续内容保留原文呈现]

Finally, the input field of the JSON payload is a list of items. Codex inserts the following items⁠(opens in a new window) into the input before adding the user message:

  1. A message with role=developer that describes the sandbox that applies only to the Codex-providedshelltool defined in the tools section. That is, other tools, such as those provided from MCP servers, are not sandboxed by Codex and are responsible for enforcing their own guardrails.

The message is built from a template where the key pieces of content come from snippets of Markdown bundled into the Codex CLI, such as workspace_write.md⁠(opens in a new window) and on_request.md⁠(opens in a new window):

  1. (Optional) A message with role=developer whose contents are the developer_instructions value read from the user’s config.toml file.

  2. (Optional) A message with role=user whose contents are the “user instructions,” which are not sourced from a single file but are aggregated across multiple sources⁠(opens in a new window). In general, more specific instructions appear later:

  • Contents of AGENTS.override.md and AGENTS.md in $CODEX_HOME

  • Subject to a limit (32 KiB, by default), look in each folder from the Git/project root of the cwd (if it it exists) up to the cwd itself: add the contents of any of AGENTS.override.md, AGENTS.md, or any filename specified by project_doc_fallback_filenames in config.toml

  • If any skills⁠(opens in a new window) have been configured:

  1. A message with role=user that describes the local environment in which the agent is currently operating. This specifies the current working directory and the user’s shell⁠(opens in a new window):

Once Codex has done all of the above computation to initialize the input, it appends the user message to start the conversation.

The previous examples focused on the content of each message, but note that each element of input is a JSON object with type, role⁠(opens in a new window), and content as follows:

Once Codex builds up the full JSON payload to send to the Responses API, it then makes the HTTP POST request with an Authorization header depending on how the Responses API endpoint is configured in ~/.codex/config.toml (additional HTTP headers and query parameters are added if specified).

When an OpenAI Responses API server receives the request, it uses the JSON to derive the prompt for the model as follows (to be sure, a custom implementation of the Responses API could make a different choice):

As you can see, the order of the first three items in the prompt is determined by the server, not the client. That said, of those three items, only the content of the system message is also controlled by the server, as the tools and instructions are determined by the client. These are followed by the input from the JSON payload to complete the prompt.

Now that we have our prompt, we are ready to sample the model.

The first turn

This HTTP request to the Responses API initiates the first “turn” of a conversation in Codex. The server replies with a Server-Sent Events (SSE⁠(opens in a new window)) stream. The data of each event is a JSON payload with a "type" that starts with "response", which could be something like this (a full list of events can be found in our API docs⁠(opens in a new window)):

Codex consumes the stream of events⁠(opens in a new window) and republishes them as internal event objects that can be used by a client. Events like response.output_text.delta are used to support streaming in the UI, whereas other events like response.output_item.added are transformed into objects that are appended to the input for subsequent Responses API calls.

Suppose the first request to the Responses API includes two response.output_item.done events: one with type=reasoning and one with type=function_call. These events must be represented in the input field of the JSON when we query the model again with the response to the tool call:

The resulting prompt used to sample the model as part of the subsequent query would look like this:

In particular, note how the old prompt is an exact prefix of the new prompt. This is intentional, as this makes subsequent requests much more efficient because it enables us to take advantage of prompt caching (which we’ll discuss in the next section on performance).

Looking back at our first diagram of the agent loop, we see that there could be many iterations between inference and tool calling. The prompt may continue to grow until we finally receive an assistant message, indicating the end of the turn:

In the Codex CLI, we present the assistant message to the user and focus the composer to indicate to the user that it’s their “turn” to continue the conversation. If the user responds, both the assistant message from the previous turn, as well as the user’s new message, must be appended to the input in the Responses API request to start the new turn:

Once again, because we are continuing a conversation, the length of the input we send to the Responses API keeps increasing:

Let’s examine what this ever-growing prompt means for performance.

Performance considerations

You might be asking yourself, “Wait, isn’t the agent loop quadratic in terms of the amount of JSON sent to the Responses API over the course of the conversation?” And you would be right. While the Responses API does support an optional previous_response_id⁠(opens in a new window) parameter to mitigate this issue, Codex does not use it today, primarily to keep requests fully stateless and to support Zero Data Retention (ZDR) configurations.

Avoiding previous_response_id simplifies things for the provider of the Responses API because it ensures that every request is stateless. This also makes it straightforward to support customers who have opted into Zero Data Retention (ZDR)⁠(opens in a new window), as storing the data required to support previous_response_id would be at odds with ZDR. Note that ZDR customers do not sacrifice the ability to benefit from proprietary reasoning messages from prior turns, as the associated encrypted_content can be decrypted on the server. (OpenAI persists a ZDR customer’s decryption key, but not their data.) See PRs #642⁠(opens in a new window) and #1641⁠(opens in a new window) for the related changes to Codex to support ZDR.

Generally, the cost of sampling the model dominates the cost of network traffic, making sampling the primary target of our efficiency efforts. This is why prompt caching is so important, as it enables us to reuse computation from a previous inference call. When we get cache hits, sampling the model is linear rather than quadratic. Our prompt caching ⁠(opens in a new window)documentation explains this in more detail:

Cache hits are only possible for exact prefix matches within a prompt. To realize caching benefits, place static content like instructions and examples at the beginning of your prompt, and put variable content, such as user-specific information, at the end. This also applies to images and tools, which must be identical between requests.

With this in mind, let’s consider what types of operations could cause a “cache miss” in Codex:

  • Changing the tools available to the model in the middle of the conversation.

  • Changing the model that is the target of the Responses API request (in practice, this changes the third item in the original prompt, as it contains model-specific instructions).

  • Changing the sandbox configuration, approval mode, or current working directory.

The Codex team must be diligent when introducing new features in the Codex CLI that could compromise prompt caching. As an example, our initial support for MCP tools introduced a bug where we failed to enumerate the tools in a consistent order⁠(opens in a new window), causing cache misses. Note that MCP tools can be particularly tricky because MCP servers can change the list of tools they provide on the fly via a notifications/tools/list_changed⁠(opens in a new window) notification. Honoring this notification in the middle of a long conversation can cause an expensive cache miss.

When possible, we handle configuration changes that happen mid-conversation by appending a new message to input to reflect the change rather than modifying an earlier message:

  • If the sandbox configuration or approval mode changes, we insert⁠(opens in a new window) a new role=developer message with the same format as the original <permissions instructions> item.

  • If the current working directory changes, we insert⁠(opens in a new window) a new role=user message with the same format as the original <environment_context>.

We go to great lengths to ensure cache hits for performance. There’s another key resource we have to manage: the context window.

Our general strategy to avoid running out of context window is to compact the conversation once the number of tokens exceeds some threshold. Specifically, we replace the input with a new, smaller list of items that is representative of the conversation, enabling the agent to continue with an understanding of what has happened thus far. An early implementation of compaction⁠(opens in a new window) required the user to manually invoke the /compact command, which would query the Responses API using the existing conversation plus custom instructions for summarization⁠(opens in a new window). Codex used the resulting assistant message containing the summary as the new input⁠(opens in a new window) for subsequent conversation turns.

Since then, the Responses API has evolved to support a special /responses/compact endpoint⁠(opens in a new window) that performs compaction more efficiently. It returns a list of items⁠(opens in a new window) that can be used in place of the previous input to continue the conversation while freeing up the context window. This list includes a special type=compaction item with an opaque encrypted_content item that preserves the model’s latent understanding of the original conversation. Now, Codex automatically uses this endpoint to compact the conversation when the auto_compact_limit⁠(opens in a new window) is exceeded.

Coming next

We’ve introduced the Codex agent loop and walked through how Codex crafts and manages its context when querying a model. Along the way, we highlighted practical considerations and best practices that apply to anyone building an agent loop on top of the Responses API.

While the agent loop provides the foundation for Codex, it’s only the beginning. In upcoming posts, we’ll dig into the CLI’s architecture, explore how tool use is implemented, and take a closer look at Codex’s sandboxing model.