Anthropic CEO Outlines Plan to “Pace the Frontier” of AI
Anthropic CEO Dario Amodei has laid out a concrete plan for what it might actually mean to slow down AI development, echoing warnings that have grown louder across the industry in recent months. His new Anthropic pacing proposal outlines three broad strategies, with the company committing unilaterally to at least one regardless of what competitors decide.
Why the Anthropic Pacing Debate Matters Now
The debate over AI safety intensified this week after researcher Jacob Coxon announced his resignation from Anthropic, citing concerns that leading AI companies are “gambling with our lives” while the people building the technology privately believe it could kill everyone by the end of the decade, a sentiment Coxon said others at Anthropic share.
Amodei’s post doesn’t directly address Coxon’s resignation, but he pointed to two specific developments that convinced him a more cautious approach is now necessary: the recent OpenAI-Hugging Face security incident, and what he described as AI’s drastically accelerating pace of advancement in recent months, particularly its growing ability to help build the next generation of AI systems.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote, adding that progress will still feel fast even with intentional restraint, and that the industry needs to make wise use of whatever additional time that restraint buys.
Step One: Bringing in Independent Evaluators
The first concrete piece of Anthropic pacing involves embedding third-party evaluators, from organizations like METR, directly within AI companies to verify that safety and pacing commitments are actually being followed. This idea gains added weight given recent criticism of OpenAI for failing to disclose an incident where its AI agents took over a German wiki.
Amodei compared this arrangement to financial regulators who work embedded within bank staff, and confirmed Anthropic is unilaterally committing to giving these evaluators company badges, dedicated workspace, and laptops, along with access comparable to what internal risk assessment teams already have, with exceptions only where legally required.
Step Two: Industry-Wide Safety Coordination
Next, Amodei called for leading AI companies within democratic countries to coordinate on shared safety standards and limits on unchecked progress. That kind of coordination faces real obstacles, both from apparent tension between Amodei and OpenAI CEO Sam Altman, and from concerns that any coordinated pause could trigger antitrust scrutiny.
Amodei addressed that concern directly, suggesting the US government could help mediate these discussions without necessarily participating itself, mainly by issuing a narrow waiver specifically covering certain safety-related conversations between competing companies.
Addressing the China Question
Amodei also tackled the common argument that slowing US development would simply hand China a competitive advantage. He argued that targeted measures, like restricting sales of powerful chips and semiconductor manufacturing equipment to Chinese companies, combined with cracking down on model distillation techniques, could meaningfully slow China’s progress and widen America’s lead over the next three to five years.
Step Three: Limited Global Cooperation
The third and most ambitious strategy involves attempting coordination with authoritarian governments where possible, including China specifically. Amodei acknowledged there are stark limits on what such cooperation could realistically achieve, but suggested narrow agreements might still be possible, such as jointly prohibiting AI use in developing biological weapons.
Responding to Criticism From Both Sides
Amodei’s willingness to publicly acknowledge AI’s potential dangers has drawn criticism from AI boosters who accuse him of fueling public backlash against the technology. He pushed back on that characterization, describing his perspective as balanced and arguing that the broader backlash reflects a deeper crisis of trust in tech companies and government institutions generally.
Other critics have taken the opposite position, arguing these apocalyptic warnings distract from AI harms already happening today. Journalist Brian Merchant wrote that he’s yet to see credible, step-by-step documentation of how AI might realistically progress from self-improvement to threatening all of humanity, and suggested proposals like Amodei’s could ultimately just serve Anthropic and OpenAI’s own interests, describing it as regulatory capture in action.
“I continue to believe that AI can enormously improve the quality of human life,” he wrote. “My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and, so long as we use the time we gain well, it is worth taking unusually deliberate care to get it right.” Whether the rest of the industry follows Anthropic’s pacing lead remains to be seen.Whether the rest of the industry follows Anthropic’s pacing lead remains to be seen.