We’ve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence (AI) and even comments from OpenAI chief executive, Sam Altman, that it might be time to “pace” AI development, while analysts wonder what becomes of the industry in that eventuality.
In a new blog post, Anthropic chief executive officer, Dario Amodei not only echoed the call to “pace the frontier,” but also outlined three broad strategies for doing so. Amodei said Anthropic is “unilaterally committing” to one of them, with Altman concurring that OpenAI would follow suit.
The debate over AI safety and alignment intensified last week after researcher, Jacob Coxon, wrote that he was resigning from Anthropic over concerns that the leading AI companies were “gambling with our lives” while the people building the technology “earnestly believe it could kill us all by the end of the decade,” a claim repeated by Anthropic officials.
Amodei’s post doesn’t explicitly mention Coxon’s resignation or his concerns, but the CEO wrote that two things convinced him it’s time to take a more cautious approach to AI development: the OpenAI-HuggingFace hack, and the fact that “AI has been advancing drastically faster” in recent months, particularly with its “growing ability to build the next generation of AI.”
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
Other AI executives seem to have reacted positively to Amodei’s post, with Altman writing, “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.” And SpaceX CEO, Elon Musk posted: “Dario is right.”
Amodei’s proposed first step would involve “embedded evaluators” from third-party organisations like METR — evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized for not reporting an incident where its AI agents took over a German wiki forum.)
Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is “something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).” That means giving evaluators company badges, desks, and laptops, and providing access “mostly comparable to what internal risk assessment teams have,” with exceptions when required by law or contracts. Altman also said this was a “good idea” and said OpenAI would do the same: “We’ll have more to share soon.”
Next, Amodei called for the leading AI companies “within democratic countries” to coordinate “common safety standards as well as limits on the rate of unchecked AI progress.” Such coordination might seem unlikely, because these companies are reportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that “for antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.”
The Anthropic CEO also acknowledged the specter of Chinese AI dominance that’s often raised as an argument against slowing development. But he said if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well as cracking down on model distillation, they could “slow China’s progress enough to widen America’s lead significantly over the next three to five years.”
Lastly, Amodei called for “global coordination,” where the United States and its allies “attempt to coordinate with authoritarian governments, to the extent this is possible.” Amodei said this would mean “cooperation with China,” and he admitted that there are “stark limits on what can be achieved,” but he still suggested there might be opportunities for agreement, even if it’s just “prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.”
With Amodei’s past willingness to acknowledge AI’s potential dangers, and with the company’s relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said he’s tried to offer a “balanced” perspective and argued that the backlash is “fundamentally a crisis of trust,” as people have become skeptical of tech companies, the tech industry, and the government.
