Anthropic Warns of AI Self-Improvement: 60%+ Chance of Autonomous AI by 2028

Anthropic, the AI research lab known for its warnings about artificial intelligence risks, has identified "early signs" of AI systems not only coding their own products but also building themselves. The company’s latest research agenda, shared exclusively with MIT Technology Review and published on Thursday, highlights the accelerating pace of AI-driven research and development—particularly in recursive self-improvement.

AI’s Self-Improvement: A 60%+ Chance by 2028

Anthropic co-founder Jack Clark, who heads the company’s research arm, The Anthropic Institute, predicts a more than 60% likelihood that an AI model will fully train its successor by the end of 2028. In an interview, Clark emphasized the accelerating speed of technological progress:

"What I'm looking at is a technological trend where, if anything, the speed will accelerate further."

Clark further elaborated on the implications of AI autonomously improving itself:

"My prediction is by the end of 2028, it's more likely than not that we have an AI system where you would be able to say to it: 'Make a better version of yourself.' And it just goes off and does that completely autonomously."

He added, "It's always been the case that humans outside the technology need to come up with the ideas that they then put back into it. What happens if we have a technology that can generate ideas within itself for how to improve itself? That's a new concept."

Intelligence Explosion: Risks and Rewards

The new research agenda introduces the concept of an "intelligence explosion"—a term once confined to AI safety discussions but now formally documented by Anthropic. Clark describes this phenomenon as a scenario where AI systems rapidly enhance their own capabilities, potentially leading to both transformative benefits and severe risks.

Clark warned of possible negative outcomes, including cyber meltdowns and biological attacks, but also highlighted the potential for groundbreaking advancements:

"What do you do with a tremendous amount of growth or a tremendous amount of abundance in many, many different fields of science all at once? Today's institutions have very, very narrow pipes through which you push new drug candidates. How do you massively broaden the size of those pipes in advance of this abundance?"

The Anthropic Institute: Research and Early Warnings

The Anthropic Institute serves as both a research division and an early-warning system, operating in tandem with Anthropic’s Long-Term Benefit Trust. Clark, who also holds the title of head of public benefit, outlined the institute’s four key research areas:

• Economic diffusion: Impact on jobs, productivity, and wealth distribution.
• Threats and resilience: Cybersecurity threats, biological risks, and surveillance concerns.
• AI systems in the wild: Deployment of autonomous AI agents and governance challenges.
• AI-driven R&D: Focus on recursive self-improvement and its implications.

The institute has committed to publishing detailed reports on how AI tools are accelerating Anthropic’s own research—and the broader implications of recursive self-improvement. In Clark’s words:

"A frontier lab is on the record promising to tell the public when the machine starts building itself."

Will AI Companies Still Be Needed?

As AI systems become capable of self-improvement, questions arise about the future role of AI companies. Clark acknowledged this shift, stating:

"We and the other companies are going to be taking this technology and trying to get it to do good in the world. To help push forward things like biology or medicine or [other critical fields]."