At Anthropic, AI is already speeding up its own improvement, an independent evaluator says

For eleven weeks, from August 12 to October 29, a METR team worked on Anthropic computers. It had access to internal data and documents and interviewed about fifteen employees. Its question: is AI already accelerating the development of AI at one of the industry's most advanced labs? Its answer, published on November 5, is: probably, yes.

METR, a nonprofit based in Berkeley, California, that specializes in evaluating the risks of advanced AI, estimates that Anthropic is improving its models about twice as fast as between March 2024 and October 2025. That holds at least for "well-defined" tasks, those that can be scored automatically. A factor between 1.3 and 3 would not surprise it. It attributes about 80% of this acceleration to AI itself, with an uncertainty it describes as significant.

How does METR measure this acceleration?

First clue: an internal Anthropic metric that aggregates benchmark results, inspired by Epoch AI's capabilities index. Since October 2025, it has reportedly been progressing 1.9 times faster than before (95% confidence interval: 1.5 to 2.8), or roughly two years of progress in one year. METR says it is "confident" that this acceleration does not come from an increase in the compute devoted to training.

The other clues are more fragile. In internal surveys, employees estimate on average that the best internal model makes them about twelve times more efficient: METR sees this only as a "very rough" indicator. Fully autonomous contributions by AI to the most important training work remain limited, even though researchers say they see "signs of life." Finally, METR sees no convincing bottleneck for the next twelve months, while admitting low confidence on this point.

What the public cannot verify

METR itself spells out the main limitation of its report. Anthropic had a unilateral right to redact non-public information, and it masked most of the elements METR considered the most telling: detailed step-by-step estimates, bottleneck analyses, forecasts about future models. Anthropic says it wanted to remove confidential information that "could accelerate other AI developers." METR says that all redactions that matter to its conclusions are described in its report, but admits it is hard for the public to judge this evidence for itself.

The report lists other caveats. METR did not have access equivalent to that of an employee: it neither requested nor obtained access to unreleased models, and had no direct access to systems. Anthropic canceled midway a component that was meant to check how its compute is split between capabilities and safety. And METR relied heavily on Claude, Anthropic's AI, to collect and analyze data. It nonetheless considers it "extremely unlikely" that Anthropic tried to deceive it. In mid-October, Anthropic had announced a broader agreement with METR that limits its redaction rights, but separate from this evaluation.

A warning threshold that may already have been crossed

The same day, METR published a second analysis, more uncomfortable for the company. Anthropic's safety policy includes a warning threshold: a doubling of the pace of progress, due in large part to the automation of research. Its August risk report stated that the acceleration remained below a factor of 2. METR judges this conclusion "overconfident": the threshold was "plausibly" crossed as early as July 15, and "about 2" would be a more accurate description. METR notes that most researchers at METR and Anthropic lean toward the threshold not having been crossed. It cannot, however, "rule out" a further major acceleration in the next six months, which could quickly lead to "high" risks.

Approached by the Associated Press, Anthropic had not responded immediately. On September 12, its CEO Dario Amodei had called on the industry to slow the pace of model improvement, and cited METR as a suitable external evaluator. It remains to be seen whether the company will embrace the diagnosis of the evaluator it invited in.