Anthropic warns AI may soon begin recursive self-improvement

Anthropic warns AI could soon start improving itself. Critics aren’t convinced

The maker of Claude wants AI labs, including itself, to prepare for a coordinated slowdown if models begin building their own successors

The companies at the frontier of artificial intelligence should be ready to slow down, one of the fastest-moving among them says.

Anthropic, the maker of the Claude chatbot , has claimed AI systems may be on the cusp of what it calls recursive self-improvement—the point at which they can design and build their own successors with little human input . The company said this could increase the risk of humans losing control of the technology.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” Anthropic said in a June 4 blog post entitled “When AI Builds Itself.”

If you're enjoying this article, consider supporting our award-winning journalism by subscribing . By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

The proposal highlights a tough problem in AI governance . A slowdown would need rival companies and governments in several countries to accept the same limits at the same time, with no treaty obliging them and competition only intensifying. That makes the warning technically important and politically fraught: Anthropic has called for the brakes in a race in which it remains a front-runner.

The speed at which the technology is developing could have “huge implications” for society, the blog post stated. The company pointed to its own operation as a warning sign. Anthropic said Claude now writes more than 80 percent of the code merged into its systems, up from low single digits before the company released Claude Code in early 2025. And Anthropic added that its engineers ship around eight times as much code per quarter as they did a few years ago. At each step of building AI, the company argued, the human role is shrinking. “We are not there yet, and recursive self-improvement is not inevitable,” the company said. “But it could come sooner than most institutions are prepared for.”