OpenAI Details ChatGPT Ad Stack: Stateful Retrieval and Auctions via LLM Tool Calls

Traditional advertising systems are built to respond to discrete units of user intent. A user searches for or views something online, the ads system receives those queries and page views as events, and the system looks for matching ads based on those events. Building an ads stack for ChatGPT was different: user intent develops across conversations and shifts over multiple turns of a conversation, sometimes without a fitting advertising moment. To support this new paradigm, we built a bespoke ads delivery system. This new type of ad stack is based on a custom, post-trained frontier LLM that makes tool calls for retrieval (identifying the top options), ranking (predicting user engagement), relevance (evaluating the usefulness of the ad), and running auctions for final ad selection and pricing.

Our ads system supports the nuance of evolving user goals by making three key design decisions: 1) implementing stateful ads sessions, meaning information is remembered across turns of a conversation and across multiple conversations, 2) relying on a custom, post-trained LLM that uses a tool call architecture to interpret a conversation’s context and underlying intent, and then effectively select, rank, and deliver ads, and 3) running the ads selection process separately from ChatGPT’s answer generation, in its own service, to keep ads from influencing answers and maintain ChatGPT’s neutral voice. We use our post-trained LLM to orchestrate the entire ads flow, determining the right ad to surface at the right time based on both the user’s current turn and conversation history. While bootstrapping a self-improving ads system like this used to take years, ChatGPT Ads reached $1 billion in annualized revenue run rate less than 200 days after launch; now, more than 100,000 advertisers have run ads on ChatGPT since launch.

How should ads work in agentic chat?

A conventional advertising request usually starts with a well-defined event. A user submits a search query or opens a webpage and the ads system receives a request. This type of system is fairly simple: it retrieves potential ad candidates, ranks them, runs an auction to appropriately price the opportunity, and shows the resulting ad to the user. ChatGPT does not naturally produce such a concrete moment.

In a ChatGPT conversation, people arrive with goals rather than a one-answer search query. They might be planning a trip, choosing software for their business, furnishing a home, or learning a new hobby. This doesn’t all happen in one turn of a conversation. ChatGPT often helps the user clarify what matters along the way.

As an example, our engineering team asked ChatGPT how to train for a race: A conversation begins with a general question about running. Several turns later, they ask about shoes to help train. The commercial meaning of the conversation changes as each constraint arrives—and this personalized context evolves across multiple sessions. A system that acted on the first message could retrieve the wrong intent entirely, potentially degrading the user’s experience and getting in their way.

Building our ads system required us to move beyond the request-response advertising architecture and build an ads stack from scratch to find potential economic opportunities while prioritizing user trust or value.

Conversation memory, without influencing ChatGPT answers

Turning every message into a standalone request, like a traditional ads system, would have carried a few risks in our chat use case: 1) misreading the user goal by oversimplifying it, 2) beginning the advertising flow too soon, before the user’s real need has become clear, 3) showing ads that get in the way of the user journey instead of supporting it, and 4) inefficiently rerunning the ads flow every time the request changes, creating noise and wasting cost. Instead of building around individual events, the ads system (consisting of retrieval, ranking, and delivery) maintains a stateful session, meaning the ads session runs for the duration of an eligible turn, remembering and updating information as it evolves. ChatGPT sends the initial request context to the ads server. The ads system follows and adapts to the developing exchange rather than relying on a single flattened query.

The ads selection process runs alongside ChatGPT’s normal response generation to ensure the ads flow doesn’t influence ChatGPT’s responses. Ads are filtered and selected through a separate system consuming the conversation under its own eligibility, quality, and delivery constraints. This important architectural boundary—different services that use separate models and separate code— lets core ChatGPT servers handle generating the organic response; meanwhile, the ads system independently evaluates whether there’s an appropriate advertising opportunity and, if so, prepares a sponsored placement through a separate serving process.

Advertising runs alongside the ChatGPT answer path.

Beyond protecting the user experience, this separation has some practical consequences. ChatGPT has no dependency on the ads system—the services are in no way intertwined and don’t share a content generation model. So if the ads serving system is unavailable or slow, ChatGPT’s organic response continues unaffected. This means the ads system can’t take down or delay the ChatGPT product. The other benefit of this separation is that it keeps the bar very high for valuable ads. Returning no ad is an ordinary, successful outcome. The system only delivers an ad if it’s clearly beneficial to users and advertisers.

An LLM interprets conversations and calls tools

The other key architectural component in our ads stack is the post-trained LLM that interprets conversations and runs the entire ads flow in an agentic loop—making tool calls to the core functions of the ads system, from the retrieval stage to ranking to auction and, ultimately, delivery. We call this LLM the orchestrator.

The first part of an ads flow is understanding user intent to start the retrieval process, which is a fast and deliberately broad sourcing of potential ads to later filter through. Receiving the conversation incrementally creates a richer but more ambiguous retrieval problem because the ads system needs to distinguish current intent from stale context. The orchestrator must interpret references to earlier messages, recognize when a user is speaking hypothetically, and avoid treating every mention of a product or service as a request to buy something.

The ads orchestrator uses the available conversation and user context, iterating to find good ads and repeatedly choosing when to call which tool with the underlying goal of finding the most appropriate ad (the ad that balances high user value and high advertiser value) for that conversation and context. If the user chooses to personalize ads, the orchestrator also considers available nonsensitive information about the user’s past conversations to find the best ad matches.

The ads orchestrator iteratively calls ads delivery tools to find valuable ads.

As the conversation progresses and the ads orchestrator gets more updated information, the orchestrator can run a fresh set of tool calls with the updated context to find better ads and return them to the client, which can then use these new ads if they arrive in time.

The orchestrator is designed to balance user value and advertiser value, always putting users first.

The ads system consists of individual tools, and the orchestrator calls them iteratively to find the best ads for a conversation. At a high level, the orchestrator turns conversations into brief text summaries, which are then turned into semantic embeddings (stored meanings that are easier for an ads system to sift through), retrieves ad candidates using those semantic embeddings, filters those candidates for eligibility (based on budget, user settings, and other preliminary checks), assesses their quality and relevance, and does a final ranking to identify high value opportunities for placement at the end of a chat response. Each ad must therefore meet a high bar on eligibility and relevance in order to be eligible for placement.

Preparing retrieval queries that capture intent

The orchestrator LLM first prepares one or more queries representing the relevant commercial intent. This can include rewriting conversation context to broaden the commercial possibilities beyond 1 search queries and then creating semantic representations, where the orchestrator distills a multiturn conversation into a summarized version for the retrieval system—retrieving ads from our database in a focused way.

Ad placements will be better for both users and advertisers if they only appear when there’s deep understanding of the user’s intent. Multiple queries can preserve different aspects of intent. A conversation about planning a trip, for example, might contain distinct needs related to transportation, lodging, or activities. The retrieval system can preserve these distinctions rather than compressing the entire conversation into one literal string.

As an example, let’s look at a potential conversation sequence about hiking shoes:

1. What should I cover on a two-day trip to Yosemite?

2. Will it be cold next week?

3. New [insert brand] product lines?

Although the user hasn’t connected the shoe brand question to the Yosemite trip, this conversation might be represented in our ads system by two queries, such as [shoes appropriate for hiking trip] and [new [brand] product lines good for travel]. This might only be the first attempt; the orchestrator is designed to iterate on these representations until it finds ads in our inventory that are highly valuable to the user.

Retrieving possible ads

Retrieval is the process of creating a deliberately broad pool of ad options. Candidates may come from different retrieval strategies and can later compete under the same serving constraints. We build multiple indices that optimize for different system and product constraints (for example, the index for product feed ads is designed to be aware of positive or negative product reviews as it fetches the best possible ads. This is not relevant to other ads that may not have reviews associated with them). Each index is optimized for some version of approximate-nearest-neighbor (ANN) search, using embeddings representing the user’s intent. This ANN search is an approximation algorithm for retrieval over large semantic indices, meaning ads are fetched very quickly, getting close to a good solution rather than looking for “the right” solution, while retaining most of the relevant candidates.

The ads system uses rewritten queries to create semantic embeddings during the ad retrieval process and produce a broad pool of ad candidates.

Filtering ineligible candidates

The goal of filtering is to eliminate all ad candidates from consideration that don't meet the bar for placement when it comes to relevance, bids, and budget. Depending on the candidate and configuration, filtering checks can include creative and campaign availability (whether the campaign or ad-creatives are active), campaign targeting (e.g., which locales a campaign is allowed to operate in), policy restrictions (compliance requirements around what can be advertised), budget availability (whether or not a campaign still has budget available to spend), pacing (whether a campaign should be spending budget now versus another time in the day), placement compatibility (what kinds of conversations this advertisement is eligible for), and advertiser and campaign level deduplication (picking an appropriate number of ads per advertiser to ensure fair participation across businesses).

Evaluating relevance and quality

Candidates that survive eligibility checks move on to a scoring phase, where the orchestrator makes tool calls to various scoring systems—for serving considerations such as semantic relevance (is the ad relevant to the user’s goal?), predicted engagement (is the ad likely to get viewed, clicked, and converted into the advertiser’s desired outcome?), historical performance (has the ad performed well when shown to users so far?). The orchestrator also filters ads to ensure relevance and safety before the auction. These systems narrow the field into a smaller candidate pool ready for auction, but they still don’t determine the final delivered ad.

Running the ads auction

The orchestrator evaluates the prepared candidates in the context of the conversation and chooses which ad candidates are plausible enough to enter the final auction. The orchestrator doesn’t directly decide which ad will be served, nor does it set a price. It nominates contextually appropriate candidates for the auction tool to evaluate.

The auction applies the final serving mechanics, including auction admission (passing a minimum threshold of relevance and value), pacing (spending ad budget more evenly), bid and auction-feature computation, quality-adjusted ranking, pricing, and final selection.

Final eligibility checks

After the auction, there are two ways an ad might fail to deliver, both specific to the ChatGPT conversational medium.

1. The ad might not feel appropriate in the chat: Even though the conversation itself was deemed safe for ad placement by integrity models (the models that decide when it’s appropriate to serve an ad), and the ad itself was deemed policy-compliant, sometimes the ad might feel inappropriate in the context of certain conversations. The orchestrator blocks these ads from being shown.

2. The ad request might not arrive in time: If the ChatGPT model is ready to respond, but the ad is not yet ready to place, the ad request times out and the ad doesn’t get delivered. One goal of the ads system, therefore, is to run as fast as possible. The ChatGPT model response time is dynamic, and we don’t delay chat responses to wait for ads to load.

If all of the above checks are met, we send the ad to the frontend for rendering. We consider this a server-side ad delivery, but we still might not show the ad to the user—and this is by design.

The final step: showing the ads to the user

In a conversational product like ChatGPT, server-side delivery of an ad creative does not guarantee an ad will be displayed; the ad must also arrive in time to be included in the current turn. The ad creative includes the title, description, destination link, and any call to action. Additionally, a model-generated preamble, which is a short, neutral introduction explaining how the ad relates to the conversation, is also returned by the ads system. Keeping the preamble separate from the creative preserves the advertiser’s message while keeping ChatGPT’s voice neutral.

The ChatGPT frontend receives the preamble and ad creative as a bundle. If it arrives before ChatGPT's response is fully generated, the frontend displays the ad below ChatGPT’s response, clearly labeled as sponsored. If it arrives too late, the ad is not shown for that turn. This is the difference between server-side ad delivery and a user-facing ad impression.

Winner selection and delivery are separate events. A selected ad becomes a delivery only after ChatGPT approves and displays the placement and the delivery is acknowledged.

Our advertising stack uses traditional logging and data pipelines, which have three main benefits: advertisers can more easily integrate their existing systems with our stack, advertisers can improve their strategies with the reporting data, and our team can find areas of the system to improve. Beyond these reporting and optimization tools, we deploy a high quality model to evaluate our delivery and train other ads models, creating a powerful self-improvement loop.

Detailed breakdowns of what happened help us understand and improve each part of the ad system in isolation. The logs processing system also records granular outcomes such as:

• The turn was not eligible for advertising.

• An ads session started but no retrieval was requested.

• Retrieval returned no candidates.

• Candidates were retrieved but failed hydration or eligibility.

• An ad was delivered and later received an interaction.

These detailed breakdowns help us improve each part of the ad serving system in isolation. Improving opportunity detection requires different evidence from improving retrieval, candidate quality, auction participation, or delivery reliability.

Delivery, serving, and quality systems observe a different part of the lifecycle. Privacy-controlled processing determines what may be used for measurement and model improvement.

High-power models evaluate and improve the entire stack

The idea behind model-based evaluation is that our strongest, highest-reasoning models watch and judge the results of our system, then improve the post-trained LLM by training it to perform better. For instance, we can use smaller, more efficient models or lower-reasoning models for orchestration or retrieval if our most powerful model gathers feedback and trains these small workers to do the best job possible.

Human feedback steers tasteful direction

We gather and act on human feedback (clicks and hides from users), as well as surveys for users and direct feedback from advertisers through our Ads Manager. We also make our team available for feedback as we build, and directly listen to what both ChatGPT users and advertisers want. This source of feedback informs our direction even more than the model’s observational feedback and is part of what allowed us to iterate toward the right product decisions quickly. The model may be able to understand user intent and think critically to deliver something valuable, but people shape taste.

What’s next

A year ago, we had no ads system and only a few people thinking about this problem space. Now we have a robust ads system powering more than $1 billion in annualized run rate, with an LLM-based orchestrator running the ads flow. Codex was a fundamental force multiplier that allowed us to bootstrap our system to achieve this major usage milestone with unparalleled speed. From the start, we set up the team and development workflows to use Codex to rapidly build and scale the system. We built guardrails for Codex to reliably and safely ship code. In parallel, we set up data infrastructure plugins for Codex to massively tighten the loop of insight gathering and landing improvements to the system, including new product features, quality improvements, and better system performance.

The combination of our system’s boundaries (grounded in our advertising principles⁠) and our LLM-based orchestrator lets us make advertising useful without treating every conversation as an ad request. Looking ahead, we’ll continue to experiment with more dynamic ad formats to increase user value (e.g. giving users the ability to talk directly to businesses through ads) and advertiser value. The stack’s self-improving architecture, built on OpenAI models with continuous feedback loops will accelerate our ability to keep improving relevance and personalization over time.