Anthropic Releases Three Metrics to Track AI Development Pace After CEO's Slowdown Call

Anthropic has published three new metrics it says could help AI companies keep tabs on how fast the technology is advancing, just days after CEO Dario Amodei stirred the industry by urging a coordinated slowdown.

In a Thursday blog post, the company detailed how it measured AI-led research and development, oversight of AI agents, and compute allocation inside Anthropic, and shared its methods in hopes other organizations will follow suit. The metrics flesh out the three-step slowdown plan Amodei published Saturday, which offered few practical details.

Amodei's weekend call drew backing from OpenAI CEO Sam Altman, SpaceX CEO Elon Musk and Google DeepMind Chair Demis Hassabis, following stark researcher warnings about AI's potential for harm. Amodei said the plan aims to temper how quickly model capabilities improve without sacrificing commercial advantage or the United States' lead in AI.

On the first metric, Anthropic said its Claude models are not operating fully autonomously for any subset of the research and development work it measured. The second involved a system to oversee and intervene in AI agent actions; roughly 30,000 agents were doing research and engineering work across its most-used internal platform at any given time.

For the third, Anthropic took a snapshot of its compute use from July 13 to July 20. It found about 6% of compute going to AI research and development was allocated to safety, while roughly 12% of compute for AI-driven research and development went toward safety.

Anthropic said these metrics best show how models are built and should complement capability evaluations, which show what models can do. Together, it said, outside parties get a starting point to assess AI's pace. "We hope to model that transparency by releasing these measurements, and we'll continue to do so," the company said.