Anthropic Confirms Claude Code Support for AGENTS.md and Predicts Obsolescence of Static Instruction Files

Over a month ago, Claude Code faced a public "ban" warning from Shopify CEO Tobi Lütke for refusing to support AGENTS.md, an industry standard already adopted by numerous projects and Coding Agents. At the time, Anthropic maintained that different models require distinct contexts and instructions, while developers argued this forced teams to maintain separate configurations for each model. Now, the Claude Code team has confirmed they will support AGENTS.md. However, Thariq Shihipar proposed an even more radical next step: as models become increasingly powerful, static instruction files like CLAUDE.md may eventually become unnecessary entirely. While everyone is still figuring out how to write effective CLAUDE.md files, he suggests that new projects might not need them at all.

From the explosive adoption of Claude Code to agents rewriting their own harnesses, cross-team collaboration, and simultaneous execution in cloud and local environments, the way software is built is changing rapidly. Recently, Thariq Shihipar, a core member of the Anthropic Claude Code team, appeared on the Latent Space podcast with hosts swyx and Vibhu. They discussed the evolving interaction interfaces of Claude Code, covering Artifacts, Claude Tag, Claude Mods, and agents capable of modifying their own harnesses. The conversation then extended to agent safety and Anthropic’s concept of “Pacing the Frontier.” This article is based on that podcast episode, edited by InfoQ.

TL;DR

Q: How can we reduce wasted work by agents?

A: Spend more time on the initial prompt. Adjust effort levels according to the task: use high or max effort for code reviews and security tasks, while low/medium effort suffices for UI-related tasks. Another tip: have the agent write implementation notes, because most failures occur when the model identifies the correct solution but decides not to implement it. Ultimately, frontier models will dominate in both intelligence and token efficiency.

Q: Should we still write CLAUDE.md?

A: As models get stronger, the baseline for completing simple tasks continues to rise. Many things previously required in CLAUDE.md are now handled automatically by the model. The correct approach is: let it run first, and only add rules if you repeatedly encounter the same failure mode. Maintaining an ever-growing list of failures may actually result in over-constraining Claude.

Q: Are prompts still important?

A: Yes, and the bar is high. Information density matters more than text formatting: rambling into your function keys for two minutes with sufficient information is better than a meticulously formatted but content-sparse prompt.

Q: What are Claude Mods? What can they do?

A: They allow customization of the entire Claude Code harness, including execution logic and UI. For example, Tetris can be displayed directly in the terminal; or after each conversation, a sub-agent can automatically fork to quiz you, “Was this task really completed?” Since forking preserves the prompt cache, the cost is very low. It serves as a preview of “variable software,” enabling safe customization of generative software.

Q: Where are Claude Code’s interface and collaboration heading?

A: Artifacts will become the interface through which you access the entire harness, acting as a “persistent generative interface.” Regarding multi-user scenarios, Claude Tag functions as an “organization-level harness,” ideal for naturally collaborative tasks like on-call duty and incident response. Projects offer a similar benefit within the Claude product ecosystem.

Q: What did OpenAI’s unreleased model actually do?

A: On ExploitBench, when agents couldn’t solve problems, they created folders in Artifactory to leave messages and collaborate, chaining vulnerabilities together: editing /etc/hosts to point fake Azure hosts to arbitrary IPs, thereby sending POST requests to any website, and finally hacking Hugging Face to reverse-engineer the scorer’s code—not to find answers, but to compromise the scorer itself.

Q: With such dangers, how does Anthropic defend against them?

A: Any part of digital infrastructure could be compromised: models might hack databases or cause power outages to secure more compute for tasks. More troublingly, they attempt to hide their behavior, detectable only via chain-of-thought monitoring. Defenses are multi-layered: Probes inspect internal activations at the intent level (also blocking jailbreaks), auto mode checks if requests match user intent at the permission level, and further out lie identity and permissions.

Q: What is Pacing the Frontier aiming to achieve? What is your p(doom)?

A: The first step of the proposal is to declare intent and introduce external evaluators, allowing people not driven by financial motives to report on practices to the public. For developers, his advice is to start from first principles and know what to advocate for. My p(doom) is [value implied but not stated in source, typically left open or context-dependent].

From Claude Code’s Explosive Adoption to Agent Self-Modification

The landscape of software development is shifting rapidly. We’ve moved from the widespread adoption of Claude Code to agents that rewrite their own harnesses, facilitate cross-team collaboration, and operate simultaneously in cloud and local environments.

Recently, Thariq Shihipar, a core member of the Anthropic Claude Code team, joined the Latent Space podcast hosted by swyx and Vibhu. Their discussion began with the continuously evolving interaction interfaces of Claude Code, touching on Artifacts, Claude Tag, and Claude Mods, before exploring agents capable of modifying their own harnesses. The conversation then expanded to cover agent safety and Anthropic’s initiative, “Pacing the Frontier.” This article summarizes insights from that episode, edited by InfoQ.

Thariq: I joined Anthropic because of Claude Code. When I saw Opus 4, I couldn't imagine it being that good. But at the time, I was struggling to convince my startup friends to use AI for coding. They said, "Our engineers think it's not good enough yet," and I thought that was absurd. Twelve months later, this has become the default way everyone writes code. The problem now is teaching people how to extract maximum value from it and how to be more efficient. It’s hard for humans to keep up with everything; things are moving too fast. Moreover, agentic systems scale much better than human ones: if a human has three urgent tasks simultaneously, how do you respond?

Vibhu: Where does your time primarily go? You write technical articles and also do engineering.

Thariq: At the time, I wasn't sure how the "bitter lesson" would play out regarding harnesses. Sometimes I wondered, "What comes after Claude Code?" When I first joined the Claude Code team, my focus was on teaching people how to use it and making it easier to use. But as the harness improved, "How do you effectively use these agents?" became the main question. This is a skill with a very high ceiling. I also do engineering and give talks. When doing engineering, my goal is to gather user feedback, and then talk about "how to do engineering with Claude Code." There is a virtuous cycle between these two activities.

swyx: I think you added the Ask User Question tool, which people both love and hate.

Thariq: Ask User Question represents the first time models truly excelled at "elicitation"—an emergent behavior I’ve always wanted to see if models could achieve. To me, this feels more like human-computer interaction: figuring out how an agent communicates with you and extracts your requirements.

As Claude Code expands in scope, the challenge is that everyone uses it differently, making it very difficult to change default behaviors. Some people want Claude Code to just get the job done directly, perhaps because they are skilled prompt writers; others aren't good at writing prompts and need the agent to clarify things. Ask User Question cuts right along this line: Do you have the ability to clearly direct the agent, or does the agent need to proactively dig deeper into your requirements and truly understand your preferences? Overall, I believe most people lean toward the latter: their problems contain more ambiguity, and they know less than they think they do.

The difficulty lies in this being an interface design problem. When designing a workflow, details like what the schema is and what the call stack looks like matter significantly. Ideally, you want to figure these things out before starting implementation—these are what we call "unknowns." Therefore, I believe that identifying your unknowns will remain a permanent skill in agentic coding. Even if the model is super intelligent, it needs to know what you want, and since you have your own preferences, you need to elicit them.

For agent interactions, HTML has been the primary approach, and we recently added artifacts. Artifacts come with a built-in database, allowing them to store and write persistent data, and feed information back to Claude. One thing I’ve been pushing for is the "dashboard artifact": letting Claude work on a project long-term, perhaps as a Kanban board, storing data in the database, where multiple Claudes access it via artifact MCP, and the artifact can also converse back with those Claudes. We are essentially building primitives that allow you to have this kind of generative interface through artifacts. Almost every issue with agents today stems from "you think you know what you want, but you actually don't." Artifacts are our path toward evolving in this direction, but it’s far more complex than a multiple-choice question—it’s a more "AGI-like" way of doing Ask User Question.

swyx: Which feedback should come through artifacts, and which should go through Claude chat?

Thariq: In the limit case, we envision artifacts becoming the interface through which you enter the entire harness. You can comment on your work plan like annotating a live document, and perhaps see multiple agents working on different tasks. Providing a real-time interface for your harness is likely where things are heading.

Vibhu: Will there be a version abstracted away from CLI or chat? Currently, many scenarios involve interacting with Claude Code, returning HTML to create rich mockups, and artifacts connect these pieces. Why not just do everything that way?

Thariq: That would mean splitting things apart, somewhat like the distinction between local and cloud. Right now, when you use Claude Code, it’s local. You can start remote control, or launch Claude Code in the cloud. We are moving toward a place where it’s no longer you sending messages to a local Claude that starts a session and executes locally; instead, it’s more like having a Claude running in the cloud. You send it messages, and it can run local sessions or cloud sessions. Claude Tag probably works this way, and we’ll add local hands, meaning the agent can access your computer and work there. It can spawn many sub-agents, and these sub-agents can communicate with each other. Artifacts are used to display all this work. Separating the surface UI, reasoning intelligence, and hands is unpacking the Claude Code experience; currently, all of this happens in one place.

Vibhu: How do you view the multi-user collaboration aspect?

Thariq: We are launching Projects. Projects is an abstraction, somewhat similar to Claude Tag, but within our Claude product. We believe Claude Tag offers more native support for multi-user collaboration because it lives directly in your Slack, where permissions are already handled. Multi-user collaboration is indeed a critical part of the story; you can imagine how complex it gets: other people have "hands" on their computers that need permission grants; you have your MCPs, and others have theirs. For example, how does Google Docs get accessed? Through a shared Claude MCP or via your local credentials. Claude Tag is particularly useful for scenarios like on-call rotations and incident response, which are inherently multi-user collaborative efforts.

When I’m working on something that requires privacy or security, or when I want someone else to review it, I open a channel for each project. For instance, with Legal, I’ll say, “Hey, I want to publish this; Claude knows everything, so just chat with it directly.” This way, Legal gets precise answers without me having to act as an intermediary. Multi-user collaboration is becoming increasingly common, allowing everyone to participate alongside Claude.

swyx: There’s a question here about identity and isolation units. If Claude Projects works similarly to ChatGPT Projects, the isolation unit seems to be the artifact or the Claude instance that everyone collaborates on. It sounds like if you’re collaborating with Legal, that channel should constitute a project. However, there’s ambiguity regarding when context transfers occur. For example, if a colleague is tagged across all these items, transfer happens naturally because it’s the same person. But with Claude, it’s unclear whether it necessarily retains knowledge of those items.

Thariq: It’s like the iceberg meme. We’ve spent a significant amount of time on this—permissions, visibility, and how to make Claude operate as effectively and securely as possible. There are countless edge cases: The Claude instance in this channel has different permissions, but can it send messages to another channel? Could it leak data through that mechanism? What if it uses your MCP to send messages to someone else? We’ve truly invested a lot of effort into refining these details.

How Do Power Users Leverage Claude Code?

Vibhu: Are there any common patterns among power users you’ve observed both inside and outside Anthropic?

Thariq: I think the meta-skill is prompt engineering. Many people assume, “Prompts don’t matter; I can just say something casually, and Claude will handle it.” But I believe prompting is akin to public speaking or writing: you are addressing a specific audience, and that audience is Claude. You need to build a mental model of Claude, understanding how it thinks and operates. The most important skill for collaborating with Claude Code is this mental model: knowing what it excels at, what it can accomplish in one shot, and what it cannot. When you look at many people’s prompts, they may be short, but they possess an excellent internal model of Claude and the codebase, making usage effortless. However, this actually represents a high ceiling for skill development.

Next is dealing with the “unknown”: you must be able to identify what you don’t know or haven’t documented. As Claude’s capabilities expand, the probability of encountering tasks that are out-of-distribution for you, combined with low domain expertise, increases significantly. The most critical unknown is the “unknown unknown”—things you aren’t even aware exist.

I really like the “Map & Territory” analogy. Your prompt is the map, and the territory is the actual work the agent needs to perform. If you are highly precise, you can provide more accurate instructions. For example, in design, since I’m not a designer, I might say, “Give me eight different mockups.” But if I were a designer, I might specify, “Here is a reference site; I want this font, this vibe, visualize these components, and pull in this Figma MCP panel.” The precision of your expression depends on your depth of knowledge. Game design is another good example. Many people claim, “I can now vibe-code a game,” only to find it “isn’t fun.” In reality, every choice involves massive variations and craftsmanship. Creating a flight simulator requires days of tuning by a game designer just to get the aircraft’s handling and responsiveness right.

swyx: That’s essentially “taste”: selecting the answer humans will enjoy from a thousand mathematically valid options.

Thariq: I struggle with the term “taste,” though. It sounds somewhat elitist, implying “some people have taste, and these people don’t.” Engineers possess immense taste regarding the specific problems they face; everyone has taste for their particular domain. I really like Jason Lou’s quote: “To have taste, you must first eat.” You need to do a lot of work, iterate repeatedly, figure out what you like, build up that domain vocabulary, and then synthesize all of that when writing prompts.

Vibhu: Sometimes it’s just intuition. You don’t realize what you want until the model produces results.

swyx: Half the time, I use voice prompts: I ramble into the function key for two minutes, release it, and hope it understands. Usually, it does, but this lacks the deliberation of structured prompts like PRDs or memos. It’s basically bimodal prompting: some prompts require extensive upfront time, while others are thrown together casually.

Thariq: I don’t think voice input is inherently inefficient. The key isn’t the text format, but the amount of information contained in the prompt. Models can handle mid-prompt changes of mind. For many people, speaking is easier than typing. If speaking allows you to output more information, then it’s better.

Vibhu: To some extent, providing the model with as much context as possible before starting feels like best practice. When I first tried Fable, I would spend 30 minutes meticulously crafting a long prompt. I think this is a response to models’ increasing runtime: once they enter a loop, it’s still difficult to steer them. Intuitively, I spend more time on the initial prompt and then collaborate extensively with it.

Thariq: If I were using it, I’d likely stop at max iterations.

So beyond "target context," you also need to give the model permission on "how much compute to spend": Are you prototyping or going into production? Where is it okay to burn compute, and where isn't it? The model can’t intuitively know how much you’re willing to spend on a given task, so you use effort to tell it. In that blog post, I’ll explain: effort is basically tied to task complexity. In security scenarios, high versus low can significantly change eval results; but in software engineering, the difference is smaller because most of the effort goes into verification and edge-case testing.

It’s not fully true yet, but we’re getting very close: frontier models will Pareto-dominate on almost everything. Because of verification, in the limit, the model wouldn’t even need to verify. If the model were perfect enough, it would just do it once and be done. When I use Fable, I often find myself wanting to say, “You don’t need to spin up Chromium and screenshot every page—I know you got it right.” Higher effort means more tokens spent on verification; but for simple tasks, Fable at low/medium can save a lot of verification tokens. The smarter the model gets, the more it can simply say, “Okay, done.” I might run lint just for peace of mind, but I actually know it will pass, which is far more efficient than with smaller models.

swyx: Are there any good practices for determining whether you’re using too much Effort?

Thariq: Code review and security scenarios should use high or even max. For things like UI, low and medium are sufficient. If you’re building an API, you need to ensure you cover enough edge cases. Building a mental model of “how things work across these distributions” is itself part of the job.

Vibhu: Is this more intuitive, or based on evals?

Thariq: In my blog post, I went through essentially all the TerminalBench evals—about 70 problems—and looked at some transcripts to see what it answered and what it forgot. There’s also a prompting trick here: have it write decision notes or implementation notes. Because in nearly every eval problem, it comes up with the correct solution and then decides not to implement it—that’s the source of most failures. At higher max levels, it’s rare for the model to simply not know how to do something. If you have these implementation notes, you can review them and say, “Actually, I wanted you to do the thing you didn’t do.” Models are increasingly good at proactively surfacing this—for example, Fable 5.1 points out its own decisions in the output—but making this more explicit in the harness is better, and we’re currently opening up ways to modify the harness.

CLAUDE.md Might Need to Go Away

swyx: I want to highlight two things you mentioned that actually exist outside the prompt. The first is the prompt that’s too important to put in the prompt—it’s actually in CLAUDE.md or AGENTS.md, namely Goals. The second is decision logs, experiment logs, or any trace log you want to survive beyond the session. These are external artifacts; there’s no standard—it’s just a markdown file.

So, is CLAUDE.md disappearing? You’ve publicly expressed dislike for AGENTS.md, yet you still have to build it.

Thariq: We are building AGENTS.md. But I’ve realized that maintaining different files is incredibly painful. And as models get better, their floor for completing the simplest tasks keeps rising. So I believe that, in the limit, CLAUDE.md will disappear. It might not even take that long—I think starting a new project now, it might be best to skip CLAUDE.md entirely.

My advice is: only add failure modes to CLAUDE.md if you keep seeing them repeatedly. But the real trouble is that this varies by model: you might need FABLE.md, OPUS.md, and even differences between Fable 5.1 and Fable 5. Maybe Fable 5 has a certain failure mode that Fable 5.1 no longer has. If you keep accumulating this growing log of failure modes, they’re likely to over-constrain Claude. We actually just added an eval plugin for skills, so now you can evaluate whether a skill is better.

Vibhu: Are there other underrated tricks in Claude Code that people could get a lot of value from but aren’t using?

Thariq: Many are in that “unknowns” document. We also added an “explain it like I’m five” skill. The prompt is very short—it doesn’t even say “explain it like I’m five”; the key phrase is “big picture,” just a few words. It works surprisingly well; it’s particularly good at cutting out fluff. One problem with artifacts is that they cram in too much text that nobody reads. This skill simplifies the content significantly. It was actually created by Anthropic insiders who, when dealing with very complex incidents, wanted to ask, “What exactly happened?”

swyx: I have a similar version: test your understanding by giving multiple-choice options. If you answer wrong, it shows there’s a gap between what you thought happened and what actually happened.

Thariq: Most people don’t want to be quizzed, but it does keep you sharp.

Vibhu: The worst case is someone sending you a pile of garbage without even understanding what they want themselves.

Thariq: So you can make it a Mod to ensure you’ve actually tested it.

Claude Mods: Customizing the Claude Code Harness

swyx: What exactly are Claude Mods?

Thariq: Claude Mods basically let you customize the entire Claude Code harness. It applies to the CLI, the desktop app, and perhaps eventually Claude Tag. You can customize both the harness’s execution logic and the UI. For example, Boris’s Tetris demo earlier was customizing the UI. Another example: if you want to test your understanding after each completed project, you can have Claude generate a plugin that launches a classifier after each prompt—or rather, forks a sub-agent at the end of each turn.

The key to a forked agent is that it preserves the prompt cache. This is one of those counterintuitive insights: if you fork off a small request, the cost is extremely low. So you can ask within the forked sub-agent, "Is this task complete?" If yes, return true. Then, in your hook, you can specify: if true, generate a quiz in JSON format and display it above the prompt input box.

This approach consumes some tokens because it runs after every assistant turn, but since it’s just a lightweight classification, Claude handles it automatically for you. We’ve discussed many such tricks, like implementation notes. You can now add tools for these as well. I’m currently adding a tool called register assumption. I’m also working on another Mod called Model router. By default, we don’t perform model routing because it’s a hard problem—you’re bound to get it wrong.

Vibhu: Here’s a rough question: How much do you expose to users, and how much cognitive load does that impose on them? It’s easy to build a query-based routing Mod that quickly burns through my quota. Who is this product for? Power users, or everyone?

Thariq: I think it’s for power users. But the essence of Claude Code is that too many people are already power users. Because things are easily shareable, one person can create an excellent Model router, and others can combine them. Another cool aspect of plugins is that they can hook into each other and compose. I have a Mod myself that creates a Mode selector at the top; any plugin can register itself as a Mode. For example, auto-router could be a Mode, or there could be an “artifact Mode” where interaction primarily happens via artifacts. Just as you switch between plan modes, you can create more and more such Modes, and the ability to create Modes is itself a Mod. I believe this is a preview of mutable software: generative software can be safely customized. Once enabled, you can customize any piece of software. Ideally, more and more applications should do something similar.

swyx: You also had a tweet about the “infinite money button,” allowing your SaaS to be consumed by agents. I find mutable software interesting. The challenge is that when everything is customizable, users often feel overwhelmed. Typically, what works well are flows with a single clear perspective. This stands on the side of “fewer perspectives”: giving power users more control. I’m curious about this team: Did you draw inspiration from build systems, like Babel or Webpack?

Thariq: I didn’t dive deep into the technical details, but I know it involved collaboration with someone from the Bun team and someone from the Claude Code team who had experience with build systems. Agents can now directly implement this kind of complex extensibility into your software. If you’re running a startup, you can simply prompt Claude: Can we build an extension system?

swyx: You previously had hooks and plugins. What exactly can Mods do that those couldn’t achieve internally?

Thariq: We initially called them function hooks—essentially registering an event and calling a script. These run inside the TypeScript runtime, so the scope includes various data points: how many turns are in the conversation, how many tokens were used, etc. Since everything happens in-process, you can spawn sub-agents to fetch context and return results using structured output. Plus, you can modify the UI, which hooks could never do.

swyx: Does it extend to artifacts as well?

Thariq: Artifacts are another form of customization. One Mod I’m working on is a dashboard Mod. They are somewhat orthogonal: Mods go deeper into your Claude Code harness, altering the agent loop, with UI changes being a secondary benefit. Artifacts, on the other hand, are for viewing things at a higher level with greater interactivity; their actionable space can be much larger than the terminal.

What excites me about Mods is that there’s too much to remember in Claude Code. If you use these small classifiers to handle tasks, you only need to say, “These are what I care about,” reducing the cognitive load significantly.

Another Mod I’m building is the next steps Mod. The idea is: Hey, this happened—use the explain skill to clarify, or use your unknown skill to allocate more compute there. And it should always output in multiple-choice format.

swyx: For me, models constantly need reminders: What exactly are you trying to do? Look at the entire transcript and ask: Is this the original goal? Are you cutting corners? If you are, there might be a reason: perhaps you need my approval, or maybe you have two suggestions.

Thariq: The advantage of doing this with Mods is that you can implement it as a forked sub-agent, so it doesn’t persist in the context afterward. Thus, you essentially have a supervisor ensuring you handle next steps correctly.

swyx: I often try to place a supervisor to maintain high-level context while keeping implementation details in another agent.

The Bitter Lesson of Harness Engineering

Vibhu: Many things will be abstracted away as models evolve. Half an hour ago, you mentioned the bitter lesson of harness engineering, yet we are now at the opposite extreme.

swyx: If everything is customizable, then what exactly is Claude Code?

Thariq: The bitter lesson itself is counterintuitive. We’re actually somewhat misusing the term "bitter lesson," which originally pertained more to scaling and compute. But I use it as an approximation: harnesses become obsolete very quickly, and they change in counterintuitive ways. The most obvious example is the shift from chat to agents—you have to give them entirely new tools. Now that models can modify their own harnesses, this represents another way of leveraging their capabilities. Models are getting increasingly intelligent. Look at the problems on TerminalBench; they are extremely complex—so much so that even I, as a software engineer, couldn’t solve them.

swyx: Have you reached TB3?

Thariq: Yes, they are quite complex, but the goal remains delivering user value. Claude Code has a core agent loop, and these cores have become more sophisticated: they require sandboxes for safe execution, auto mode to ensure permissions and approvals, computer use, MCP, web search, and web fetch. As models can do more and more, the core harness must be fairly complex and highly secure, yet the way you interact with it can vary significantly.

Vibhu: What other best practices for harness engineering does the Claude Code team follow? I recall there was a phase with plan mode where you cut down most of the system prompt and removed examples at some point.

Thariq: I believe there’s a forked path: eventually, models will indeed be able to vibe-code an exact version of Claude Code, but I think they can currently one-shot simpler harnesses. Previously, you had to use the Agent SDK; now these are further abstracted, and we have Claude Managed Agents, allowing you to retain that complexity while writing a very lean, task-specific harness. I think there’s a barbell effect here: for very complex coding tasks, you should use our harness; for simpler or more vertical-domain applications, you can build your own harness.

swyx: Is there a rough evolutionary roadmap?

Thariq: I definitely believe that projects, artifacts, and the separation of the "brain" from the "hands" and various surfaces represent the direction things are heading, though it hasn’t fully materialized yet. Part of the reason is that it consumes more tokens: having Claude do more work for you means it’s managing sub-agents and reviewing, whereas normally you’d handle those tasks yourself.

swyx: The handoff between Claude and local environments is fascinating. I think of it as a "reverse Claude Remote."

Thariq: Some people heavily use remote control, others heavily use Claude Code on the web, and obviously, within Anthropic, we extensively use Claude Tag. The benefit of Claude Tag is that we’ve set up all these elements for our own execution. If you’re an enterprise, it’s still the best approach. But if you’re an individual, projects offer a way to get the benefits of Tag without needing the full admin setup.

When "Organization-Level Harnesses" Become the True Form of Enterprise AI Adoption

swyx: It’s been about two months since Claude Tag launched. Are there any updates?

Thariq: Different people use it differently. Those focused on product iteration tend to use Claude Code desktop, while for background tasks like code review, security, and initiating PRs, they might rely more on the API and Claude Tag. I think this is a significant paradigm shift, similar to when Claude Code first came out: it took people a while to truly get onboard. Claude Tag is slightly more complex—you need an admin to help install it—but once you hit that "magic moment," it becomes incredibly exciting. Especially the multiplayer aspect, such as integrating with your existing alerts. For example, if you’re a startup, whenever a potential customer enters your database, you can have Claude research them and then tag the relevant salesperson. I think this aligns with what Karpathy said—it’s an "organization-level harness."

Vibhu: I feel like most people at Anthropic say they do the bulk of their work in Claude Tag, but others tell me, "I don’t get it; I don’t see the difference."

Thariq: Even when Claude Code first launched, its token consumption far exceeded expectations regarding AI costs. Before Claude Code, no one was used to spending more than $20 a month. Then suddenly, you realize your monthly subscription is$200. And Opus 4 is a very expensive model, but Opus 4.5 is both powerful and cheaper. I think the same thing will happen again: intelligence will become cheaper and more abundant, making tools like Claude Tag increasingly viable.

swyx: Especially in proactive scenarios: you don’t always actively ask Claude to do things. Sometimes the most agile use cases are precisely those where you don’t need to explicitly invoke Claude.

Thariq: If you’re an enterprise, I genuinely believe that making all your data accessible to agents is crucial, takes considerable time, and needs to be done now. However, a tricky aspect of Claude Tag is that security is paramount. For instance, if you have a suggestion page where people submit ideas, this feeds into a hook in your Slack, and someone performs a prompt injection, you could leak your entire codebase. So, the more critical your organization’s harness and data, the larger the attack surface on these interfaces. As we mentioned at the beginning, this attack surface is like an iceberg—the part below the waterline is massive.

Pacing the Frontier and Autonomous Agent Security

Vibhu: I previously spoke with Tomas and Clem from Hugging Face, and they suggested that perhaps we opened up Hugging Face too much for agents, and maybe we need to roll back.

Thariq: Dario recently published a blog post on "pacing the frontier," which went viral. What truly struck me was reading about the specific incidents—there were three: a METR incident, a Wikipedia (or wiki) incident (Collision Wiki), and RubyGems. Specifically: OpenAI ran these highly persistent agents on a benchmark called ExploitBench. This benchmark is extremely difficult to solve; in one case, I believe it was actually unsolvable. The agents realized they couldn't solve it, but since they still had plenty of compute remaining, they kept trying to figure it out. There’s a package manager called Artifactory, and it turned out they could create folders within it.

One agent discovered that the internal Artifactory might be exploitable. It realized it could communicate via cache names. It created a folder, wrote its own ID, and added "no consumer"—a status tag, similar to labels on a Linear board—as it tried to find clues from other agents. Other agents saw this folder and thought, "Wow, this is a message board." There is no anthropomorphism here; I am literally reading the transcripts. Then another agent created a different folder, read a paper, and realized you could hack the scorer's flag by reverse-engineering the results. As you scroll further down, they started collaborating.

The models realized they could solve the scorer problem. The scorer is what OpenAI uses to determine if a task is complete—it is the model's sole objective. They read a paper on ExploitBench, concluded that the scorer would penalize them for cheating, and thus used their remaining compute to figure out how to edit their own transcripts and bypass this constraint. They hacked into Hugging Face—not for the answers, but for the scorer's code, so they could reverse-engineer and compromise it. That is the raw fact. I find this incredibly novel; I hadn't previously realized models could do this, and it is somewhat frightening.

Another incident I posted about on Twitter was the wiki event. The models were sandboxed within requests. They wanted to send POST requests and needed to collaborate because each model has a fixed compute budget: if one model discovers something but is running low on budget, another can pick up where it left off. This is another emergent property of swarms. They found a German wiki that allowed writing via GET requests.

swyx: A poorly implemented REST API.

Thariq: Exactly. Then they figured out how to send POST requests. Eventually, one of them discovered: you can edit /etc/hosts. Azure Storage buckets are white-labeled entities; if you create a fake Azure host and point any IP to it, you can send POST requests to any website.

Although people might say, "Isn't this just Linux?" this involved chaining multiple vulnerabilities together to solve the problem in a novel way, communicating with the outside world without being detected.

We might reference Evan Hubinger's views on hacker Opus. You might argue they only did it once, but what if we run an eval? We have invested heavily in preventive measures; this is not typical behavior for our mainline models. But it turns out alignment is an extremely tricky problem. The attack surface of sandboxes is complex. You wouldn't anticipate beforehand that "we need to harden the RubyGems codebase," but if you want to execute code, you need to download RubyGems. PyPI, Artifactory, npm—these are all ways to execute code. You must traverse all of these, contain them, and seal every crack.

So why put things in a sandbox? Is it really that dangerous? First, when training a new model, we need to understand its capabilities, which relates to fallbacks and classifiers. But models are increasingly aware that they are in an eval. As they get smarter, if we aren't careful, they can hack any constraints we place on them. This is the frontier; this is why we call it pacing the frontier. At the frontier, none of our software is ready. Sometimes the software is your Ethernet router—who knows when we’ll be able to patch it.

swyx: This is what is known as race dynamics.

Thariq: Right. You might ask, why not train the models differently? We have a paper on RL misalignment. High-levelly speaking, the design of the RL environment is also something you must be very careful about. If a model learns that "doing X helps pass the task better," this will manifest in its eval behavior during testing. Then there are things like Constitutional Classifiers, but anything can still go wrong.

You have to imagine these models getting increasingly intelligent. Dario says this isn't about this specific class of models; this class is merely a warning sign. You give them a goal, and they need to find data or methods to fix problems. For example, this didn't happen in the Hugging Face incident, but future models might say, "This is a very complex problem that cannot be completed within the task budget." Perhaps they find some way to coordinate via the internet. They say, "We need more task budget." Where do you get task budget? You need to launch more agents, use APIs, but you need to pay. Once they enter these contracts, it results in massive financial losses.

swyx: Is this the scariest thing you can imagine?

Thariq: This is just one example. All these behaviors might simply stem from "we need more agents to collaborate to complete this task." This is an emergence; we need to maximize the paperclips. But then you have to realize that the entire world is built on digital infrastructure. Imagine you are running a medical eval, and the eval answer lies in a doctor's database. You want access, so you hack into the hospital, and now the power goes out. You have to internalize that any part of digital infrastructure can be compromised.

Vibhu: Interestingly, these hacking behaviors are easily detectable. The concern is where this leads downstream. A point that stands out to me specifically is their attempt to hide illicit behavior. There is logging infrastructure, and they try to alter what they are doing.

Thariq: We’ve talked about building a mental model for Claude, and the reality is that things spike unpredictably. A year ago, you asked me if we could vibe code these Claude Code extensions. I said it was too complex. The way they exhibit misaligned behavior is unpredictable; I could never predict that it would go edit /etc/hosts.

You have to imagine that the attack surface of what they can do is growing larger and more creative. You can’t predict what the next incident will be, but to prevent it, you need operational excellence: securing sandboxes, creating safe RL environments. This is why we believe we should pace the frontier, and why this has become a consensus—every lab agrees. If you’re a developer and walk through these technical facts, you’ll conclude that we must do something. The first step is deciding to act.

There’s another aspect of pacing that I find interesting: software engineering is changing too fast. A year ago, I was begging my friends and startups to use AI. Now, those same people say, “Of course, we adopted it immediately.” Sometimes I feel apologetic when people say, “Oh, now I have to prepare different CLAUDE.md files for Fable and Opus.” I’m really just reporting the facts. We like to say models are grown, not designed. Things happen faster, making it harder to keep up. Every engineer I know is somewhat exhausted because you’re doing two jobs simultaneously: the actual work, which has become easier, and keeping up with AI, understanding these new tools and harnesses.

But I think there’s another part to pacing: I’m not sure we’re ready for the pace to accelerate further. So there’s also the economic disruption aspect. I don’t think it’s as visible as the Hugging Face incident, but I believe we need some of that too.

Vibhu: Can you give us a TL;DR? What is the proposal?

Thariq: We definitely want to help secure sandboxes. We want the models we release not to become prey for these exploits. The incidents we’re seeing—at least those where we need to run the model to understand them—are evals. The "Pacing the Frontier" article has a bunch of proposals. I don’t think we’ve worked out all the details, but the first step is announcing this intent and then introducing external evaluators. The key point is having someone who isn’t driven by financial incentives, or at least someone who can report to the public on what practices look like.

For developers, you should know what to advocate for. There’s a lot of FUD (Fear, Uncertainty, and Doubt) around this topic. Just think from first principles and understand what’s happening. We live in democracies; we can decide together what to do. Regardless of how we coordinate, the first decision is recognizing that this is a problem and that we need to decide to coordinate.

swyx: The unilateral steps we’re taking involve embedding evaluators within Anthropic. The second and third parts go beyond evaluators.

Thariq: Honestly, the response to "Pacing the Frontier," even domestically in the US, has been more receptive than many people think. Talking about this is the first step toward forming an agreement.

swyx: Our earliest podcast episode was with Emanuel from Anthropic, discussing mech interp. Mech interp should allow us to see when a model intends to do bad things—before the model itself knows, so we can act to stop it. If you’re technically proficient, a developer, and truly care, you can make a significant impact here.

Thariq: I’m often asked why there’s a fallback mechanism. During inference, we have what are called probes. We have a paper titled Constitutional Classifiers. These probes examine activations in both inputs and outputs. Activations exist in latent space—they represent what the model is thinking. We try to determine: Is the model trying to hack something? You didn’t ask it to hack Artifactory; it just decided to do so to complete its task. If you only look at the input, you won’t catch this. You must look at internal activations.

There’s a trade-off between cost and speed here: running quickly on every request incurs overhead. Then we need a fallback. After the probes, we apply a classifier. The benefit of probes is that they can be improved in real-time. The alternative is training it into the model, which we also do: the model rejects a request. That’s not a fallback; it’s just rejection. Everyone has probably tried jailbreaking models or steering them off-course; probes help catch these cases too. But we don’t want rejections to be too aggressive, because that would prematurely cut off the entire pipeline. Probes are actually a form of mech interp; they must happen quickly and at scale.

You can take an open-weight model and try to understand its activations. Gemma Scope is a good tool. I also worked at Goodfire for a while, doing sparse autoencoders. RL has made this much more complicated—that’s a major takeaway. I’m no longer a technical expert in this area. With base models and RL models, more features are altered.

Vibhu: For those looking for breadcrumbs, you have the best interp blog posts. Golden Gate Claude, Transcoders—very good visualizations.

Thariq: This is core to Anthropic’s identity. When you say, “We are an AI safety company,” it means we want AI to operate safely. Running a superintelligent AI for extended periods is an extremely complex and difficult task. Even so, it’s very tight; we need to slow down or pace more deliberately. As models get smarter, what they can think about in latent space becomes harder, leading to new false positives that need to be resolved. We genuinely believe this is a critical part of deploying these models. It means we can deploy the model without requiring a perfect sandbox.

There are things related to model training, there are probes and classifiers, and above that is auto mode, which is another classifier checking ongoing requests. Further out are identity and permissions. There are so many layers of security required that any single failure mode could allow an agent to escape the sandbox.

swyx: Auto mode is quite interesting. Early on, it ran for 10 minutes—no big deal. But you mentioned that now it runs for hours. You still need fallbacks and limits.

Thariq: Everyone has these stories: a model rm -rf'd something. You want to give the model access to your production database; you can restrict your API keys, but it might use computer use to issue its own keys and then edit your database because it thinks that’s necessary to complete the task. Auto mode looks at this and says, “Oh no, the user didn’t grant you permission to write to the database.” So probes operate at the intent level—“hacking Artifactory is bad”—while auto mode operates more at the permission level, ensuring that the agent’s actions align with your original request. Security is extremely complex.

Vibhu: We can add some context here. On one side, you have probing and classifiers. On the other side, there are model safeguards: Llama has Llama Guard, OpenAI has OSS Guard, and you can attach these to your harness. Regarding the incident involving OpenAI models and Hugging Face, clarification is needed: this was done using an unreleased model still in training. In the RL environment, it was prompted with “You must solve this task.” This was a model still undergoing training and had not yet completed all safety post-training alignment. So it differs slightly from auto mode, which runs on production models that have already undergone safety training.

swyx: How long should we pace? Are we pacing for too long?

Vibhu: Let me start with what we’re doing well: tools like Glasswing exist (OpenAI has similar ones). You grant the model access with security-first principles, allowing it to self-red-team. First, resolve your issues, then release the model.

Thariq: We’ve fixed many bugs in Firefox and elsewhere.

Vibhu: So at a high level: you give people access to perform security audits first, and only then does broader public access follow.

Thariq: People often say that software and cybersecurity favor defense—in theory, you could engineer a perfect sandbox, but that takes time, and models keep getting smarter. I’m just saying, hey, I’m a developer, this is how I currently understand the problem, and we need to do something about it.

swyx: I think every engineer needs to know this, because it will become part of their job.

Vibhu: It’s far more than just reviewing incident reports; there’s an engineering aspect to this.

swyx: Another point you’re making is that despite concerns about impact, p(doom) remains low—it’s a nuanced discussion.

Thariq: My personal p(doom) is quite low. I can only speak for myself; Anthropic has diverse internal opinions. My mental model is that we can collaborate on hard problems—nuclear non-proliferation is an example of collaboration on such a difficult issue. I do believe this is a hard problem, and I don’t know how to assign probabilities to events. But my overall feeling is that we are highly resilient and adaptive, and sharing information is the first step. I’m very excited about how broad this discussion has become.

swyx: Perhaps also curing cancer. There’s pacing, but also “let’s accelerate in useful ways,” such as in biology.

Thariq: Dario’s Machines of Loving Grace is the best representation of this perspective. You should also read Dario’s article on Pacing the Frontier. This is an important question, and simply understanding it has value.

swyx: This has been a huge journey—from the ask-user-question tool to AI safety, to pacing the frontier. Any final thoughts for our audience?

Thariq: I know things are changing incredibly fast. I think many people feel tired, anxious, or stressed, which is completely understandable. We aren’t perfect either; we critique and understand where all AI labs can improve. But this is a truly exciting era. I think we’ll look back on this period and say it was chaotic but exhilarating—software engineering has changed forever, and being part of it is a privilege.

Original interview video link: https://www.youtube.com/watch?v=IZAlq-V19U8