METR Finds Anthropic Model Progress Roughly Doubled, Largely Driven by AI-Assisted R&D

Baoyu (@dotey) · 2026-11-05 22

METR Independent Assessment: Anthropic's Model Capability Progress Rate Has Roughly Doubled, Much of It Likely Attributable to AI Participating in R&D

The AI safety evaluation organization METR released a report on November 5, concluding that as of October 2026, Anthropic's rate of improving model capabilities on "well-defined tasks" is approximately 2 times that of the period from March 2024 to October 2025. METR says this judgment is highly uncertain, and that the actual figure being 1.3x or 3x would not be surprising.

METR is a nonprofit organization that studies whether frontier AI poses a risk of losing control. This report assesses "AI R&D acceleration": whether AI companies using AI to write code, run experiments, and train next-generation AI could cause capabilities to progress faster than historical trends. The concern is that the faster AI is built, the more capability growth may outpace what alignment and control measures can handle.

[Conclusions First]

METR gave three conclusions, all of which are the team's subjective judgments, and there is internal disagreement within the team:

• Speed: The rate of progress is approximately 2 times the past rate, with a range of 1.3 to 3 times. • Cause: Most of the acceleration likely comes from Anthropic's extensive use of AI for R&D, which METR estimates accounts for about 80%. METR is uncertain about this, and there is no consensus among Anthropic employees either. For the acceleration wave at the end of 2025, factors other than AI may have played a large role. • Going forward: METR has not seen obvious bottlenecks, nor has it seen significant diminishing returns, so acceleration could potentially increase further over the next 12 months. METR also notes that its perspective is limited and it may have missed bottlenecks.

The report also contains one detail: compared with before, Anthropic now keeps its latest models internal for a longer period before releasing them externally.

[How the 1.9x Figure Was Derived]

The most concrete evidence in the public report is test scores. METR used AECI, the Anthropic version of the Epoch Capability Index, a methodology proposed by the research organization Epoch AI that combines a set of test results into a single overall score to make it easier to see how capabilities grow over time.

Since October 2025, the AECI growth rate of Anthropic's models has been 1.9 times the previous rate (95% confidence interval 1.5 to 2.8 times). In other words, the progress of the past year is equivalent to the progress of the previous two years, roughly the distance from Claude 2.1 to Sonnet 4.5. This figure is higher than the 1.7x reported by Anthropic in the Haiku 5.5 system card, mainly because METR included three unreleased internal models.

This number is heavily influenced by methodology. Under a different statistical assumption, the result ranges from 1.1x to 2.9x; when other companies' models are also included in the calculation, it is only about 1.4x. The latest internal models score above 50% on every test that has been administered, and hard problems are running out, so METR worries that it will be difficult to continue testing this way in the future.

[How Much Did AI Actually Help]

One thing METR can determine is that this acceleration is unrelated to faster growth in training compute. Many details regarding other factors such as hiring, data, and algorithmic breakthroughs were redacted.

METR and Anthropic interviewed quite a few researchers, and people's accounts of how much AI has accelerated their own field varied widely, roughly clustering around 2x. Anthropic also held emoji votes on Slack internally, asking employees to estimate how many times faster the latest model made them, and the average self-assessment in October was about 12x. METR considers such votes to be only very rough evidence, and cautions that an individual working several times faster cannot be directly converted into the company as a whole progressing several times faster.

Employees also said that AI currently mainly serves as an assistant, and the most important parts of training models still cannot be fully handed over to AI to do on its own. However, some researchers said they have already seen signs, and that AI making important contributions independently may emerge soon.

[Most of the Evidence Was Redacted]

METR submitted a complete report of about 100 pages to Anthropic. Under their agreement, Anthropic can unilaterally request the removal of any non-public information in the public version. Anthropic removed most of the evidence that METR considered most persuasive, including acceleration estimates for each stage of training, estimates of algorithmic progress, how much credit belongs to compute versus data, and predictions about future models.

Anthropic's rationale is that these secrets could help other AI companies accelerate. METR agrees that the removed content falls into this category of information, but did not evaluate whether the redactions were reasonable, on the grounds that it is not suited to weigh corporate commercial interests against the public's right to know. METR says the result is that the public cannot verify the evidence themselves and must rely more on METR's judgment than would be ideal. At the end of October, Anthropic also tightened its redaction standards once, delaying the public report by about 5 days.

The contract originally also included verifying Anthropic's compute allocation, to see how much compute is used for improving capabilities and how much for safety. Halfway through the project, Anthropic canceled this part, at a time when METR had not yet reached substantive conclusions. Anthropic's explanation was that it was temporarily focusing on risk assessment and would return to it later.

This assessment ran from mid-August to the end of October, with METR investing the equivalent of 2.5 full-time staff, and it did not receive payment from Anthropic. It is a separate matter from the program Anthropic announced on October 15 granting METR "employee-like access." For this assessment, METR had to actively request data, and Anthropic could approve or refuse, and METR says it may have missed information whose existence it did not know about.

[Anthropic's Response]

Anthropic co-founder Sam McCandlish is responsible for most of the company's safety work. Sam said on X that day that many of METR's conclusions align with his own views: AI is accelerating Anthropic's research, but it is hard to say by how much; AI cannot yet independently make the most important contributions to training models, but by default the speed may increase further.

Regarding the redactions, Sam acknowledged that a lot was removed, making it harder for outsiders to verify METR's work. Sam's rationale is that disclosing exactly what is driving the acceleration involves both commercial secrets and amounts to giving others a roadmap for acceleration, which would worsen the risks METR is worried about. Sam believes this trade-off is the right one, and Anthropic is continuing to evaluate the pros and cons.

Sam said that in the coming weeks, Anthropic will publish its own report analyzing how fast internal AI progress is, where it stands relative to the RSP, and new safety measures. The RSP is Anthropic's "Responsible Scaling Policy," which stipulates that once model capabilities reach certain thresholds, corresponding levels of safety protections must be in place. Acceleration of AI R&D is one of the key risk indicators in the RSP, but METR did not specifically assess against the RSP thresholds this time; which level Anthropic is currently at will have to wait for its own report.

METR report: https://t.co/8DDcuYStCz METR tweet: https://t.co/Ds4LCsbEOF Sam McCandlish's response: https://t.co/oZKTZ8RB13 Anthropic's statement on independent evaluators: https://t.co/wl8MyzUfYa

↳ Quoting @METR_Evals (METR): Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.