AI firm calls for global halt to development while pushing its own tech forward

Anthropic, the company behind the Claude AI model, is urging governments and industry leaders to consider a worldwide pause on artificial intelligence development — even as it continues to ramp up its own capabilities and reportedly embeds engineers inside the U.S. National Security Agency.

In a lengthy blog post Thursday, Anthropic described how its Claude system is getting better at improving itself, a concept known as recursive self-improvement. This is the idea that an AI could one day design smarter versions of itself without human help. Many safety researchers see this as a potential tipping point that could lead to superintelligent systems with unpredictable consequences.

The company warned that if this trend continues, AI could eventually become capable of fully autonomous self-improvement, raising the risk that humans might lose control. To address this, Anthropic said it plans to bring together policymakers, researchers, civil society groups and other AI companies for discussions.

But the call for caution comes with a twist. According to the Financial Times, Anthropic has placed engineers inside the NSA to help the spy agency use its Mythos model for offensive cybersecurity operations. Critics say this undercuts the company’s safety messaging.

“Anthropic might appear warm and fuzzy, but their definition of AI safety is narrow,” said Steven Murdoch, a professor at University College London. “Supporting offensive capabilities has never been something they’ve spoken against.”

Others question whether the blog post signals any real breakthrough. Murdoch noted that while AI capabilities are steadily improving, nothing in Anthropic’s post suggests a sudden leap forward. The company’s main evidence: more than 80% of code now merged into its systems is written by Claude, and the AI is getting better at running experiments and proposing its own coding tasks.

This isn’t the first time Anthropic has sounded alarms. Two months ago, it announced — but refused to release — a model called Mythos, claiming it was too dangerous for public use. Some experts dismissed that as a marketing move.

Anthropic recently filed for an IPO that could value the company at $1 trillion.