Anthropic warns investors its AI could soon speed up its own progress

An IPO prospectus is usually a catalog of legal and financial risks. Anthropic's, the company behind the Claude AI, has contained a warning of a different kind since November 2. In the revised version of its filing (Form S-1/A) submitted that day to the U.S. securities regulator, the SEC, the company writes that its internal estimates indicate it could "imminently" reach a threshold it set for itself: that of "dramatically accelerated" progress thanks to AI.

TechCrunch spotted the passage on November 2. It appears in the risk factors of a filing that accompanies Anthropic's IPO plan.

What the prospectus says exactly

According to excerpts published by TechCrunch, Anthropic acknowledges using AI in its own research. It warns that "using AI to help build more capable AI could also pose a considerable safety risk." The company also admits it has not always spotted accelerations in its pace of development in time: in the past, it identified them only after the fact.

The threshold in question comes from its "responsible scaling policy," the document that sets the precautions to take depending on the capabilities of its models. According to Gizmodo, it roughly corresponds to a doubling of the usual pace of progress, plausibly attributable to AI. Gizmodo adds that crossing it would require the company to significantly strengthen its measures, including security. The prospectus links this threshold to what is known as "recursive self-improvement": the point at which an AI contributes decisively to designing the next one.

No model has crossed the line, the company says

The text sets its own limits. Anthropic states, according to TechCrunch, that it has not designated any of its current models as exceeding this threshold and that it is continuing to analyze the data. It also writes that it does not expect to be able to let Claude autonomously design and build the next generations safely, without "meaningful human oversight or understanding." It is considering changing its policy "to keep humans in charge."

None of this rests on an outside assessment. These are internal estimates, which the company does not publish.

Sincere warning or pitch to investors?

That is the whole ambiguity. A prospectus serves both to inform and to sell, TechCrunch points out, citing a post by a user on X who argues that announcing an AI able to improve itself can also amount to promotion. Gizmodo calls the notion of recursive self-improvement "rather fuzzy." The site notes that it was developed in online circles tied to rationalism and effective altruism, and suggests the warning may be aimed as much at cautious investors as at a fascinated public.

For a company seeking capital, writing that its technology is nearing such a milestone is far from trivial. The statement is legally binding: an omitted risk factor can lead to lawsuits. But it also flatters the company's valuation. No outside researcher has so far been able to verify the measurements it rests on.

The next step to watch: the new version of the responsible scaling policy that Anthropic says it is considering. It will show whether humans really stay "in charge," and by what verifiable criteria.