Anthropic, the AI safety company behind the Claude models, is urging global tech giants and governments to slow down advanced AI development before it spirals beyond human control.
The warning, published in a report and blog post titled When AI Builds Itself, is rooted not in speculation but in hard internal data that paints a striking picture of a technology accelerating beyond its own architects’ expectations.
As of May 2026, more than 80% of the code merged into Anthropic’s own production systems was authored by Claude, the company’s AI assistant, a dramatic leap from the low single digits recorded before the launch of Claude Code in early 2025. In plain terms, the AI is now largely building itself.
Anthropic engineers, on average, now ship roughly eight times as much code per quarter as they did between 2021 and 2025, a productivity explosion driven not by additional human hires but by AI doing the heavy lifting.
In one concrete example, Claude shipped over 800 fixes in a single month that reduced a class of API errors a thousandfold.
These figures form the empirical backbone of Anthropic’s alarm. The company is not crying wolf based on abstract philosophy; it is sounding the alarm based on what it is observing inside its own walls.
At the heart of Anthropic’s concern lies a concept that AI researchers have long debated in hushed tones: recursive self-improvement. This is the threshold at which AI systems could design and develop their own successors without meaningful human intervention, a feedback loop of ever-increasing machine intelligence that humans may struggle to redirect or stop.
Anthropic was careful to note that this threshold has not yet been crossed and is “not inevitable,” but warned it could arrive sooner than most governments and institutions are prepared to handle.
Jack Clark, Anthropic’s co-founder and head of policy, put it in characteristically stark terms during an interview with Britain’s BBC Newsnight on Thursday. “You want the option to be able to take your foot off the gas and put your foot on the brake,” he said. “Right now, it’s like the AI industry has a gas pedal, but it doesn’t have a brake pedal.”
He added that with Claude currently writing roughly 80% of its own underlying code, reaching 100%, the point at which the system is entirely self-generating, “is possible within two years.”
Anthropic was explicit that a unilateral pause by any single company would be strategically futile, amounting to little more than an act of self-sabotage in a fiercely competitive race. Any meaningful slowdown would require leading AI developers at or near the technological frontier to stop under identical, verifiable conditions.
That is a geopolitical ask of almost staggering ambition. Anthropic compared the challenge to nuclear arms control treaties but acknowledged it would be even harder to enforce AI model training is far easier to conceal than a missile silo, and the temptation for any one nation or company to quietly press on would be enormous.
The company announced plans to bring together government officials, scientists, advocacy groups, and competing AI firms in the coming months to begin designing a workable framework.
The proposal lands in a Washington that is deeply conflicted on the question of AI regulation. US officials and tech executives have repeatedly argued that any slowdown risks ceding strategic ground to China in what many view as the defining technology competition of the century.
President Donald Trump said he discussed the possibility of cooperating with Beijing on AI safety issues during his recent visit to China, a remarkable diplomatic opening, however preliminary.
He also signed an executive order this week granting the government a 30-day window to conduct preliminary reviews of the most powerful American AI models before their public release.
Perhaps most tellingly, the White House has acknowledged the extraordinary capabilities of Anthropic’s most advanced model, Claude Mythos, a system so powerful in its cybersecurity applications that it has not been released to the general public and is currently deployed only to a small number of vetted organizations under Anthropic’s Project Glasswing initiative.
Not everyone in the technology world is moved by Anthropic’s warning. The company has faced sustained criticism from competitors and some officials who argue its fixation on worst-case scenarios is overblown and that its safety-first posture is, in practice, a competitive strategy dressed up in altruistic language.
The proposal is also unlikely to find a sympathetic ear in Elon Musk, whose artificial intelligence venture xAI stands to benefit enormously from continued breakneck development.
The hotly anticipated stock market debut of his SpaceX company, which owns xAI, is widely expected to make Musk the world’s first trillionaire, a milestone that depends heavily on investor confidence in AI’s unbounded growth trajectory.
Beneath the geopolitics and the corporate rivalries, Anthropic’s report raises a question that cuts to the philosophical core of the AI moment: at what point does the technology cease to be a tool and become something else entirely?
“The evidence suggests that the human role is narrowing at each step in the AI development process,” the company said.
Anthropic’s head of internal research, Marina Favaro, and Clark argued in the report that a pause would give the world the breathing room it needs to adjust to the pace of AI’s expansion, time for legal systems, democratic institutions, scientific understanding, and basic ethical frameworks to catch up with a technology that is, by Anthropic’s own admission, already racing ahead of them.
Whether the world will take that offer remains, for now, an open question. But the company that built one of the most powerful AI systems on Earth is now telling anyone willing to listen: we may be moving faster than we know how to stop.
WHAT YOU SHOULD KNOW
Anthropic, one of the world’s leading AI companies, is sounding a rare and urgent alarm: the technology it helped build may soon be capable of improving itself without meaningful human oversight, and the world is not ready.
Backed by striking internal data showing AI now writes over 80% of its own code, the company is calling for a globally coordinated pause on frontier AI development before that window of control closes permanently.
Humanity currently has a gas pedal for AI, but no brake. And without one, the race between nations and corporations could carry us past a point of no return before anyone agrees to slow down.
