In an unprecedented development, 1,200 CEOs, CTOs, programmers, scientists, and AI safety officers sent a letter to Congress asking the government to help them build the tools needed to slow down AI development if it ever becomes necessary.
The statement, titled “Pacing the Frontier,” was signed by the most powerful and knowledgeable AI executives and employees in the business.
According to Fortune, the statement was signed by "Anthropic CEO Dario Amodei and several of the company’s co-founders, alongside OpenAI chief scientist Jakub Pachocki, Meta chief scientist Shengjia Zhao, and Google DeepMind’s head of AI safety and alignment, Anca Dragan."
Even AI's most enthusiastic booster and advocate for minimal government intervention, OpenAI's Sam Altman, told Axios on Wednesday that he's discussed the "need" to slow AI development with White House officials as models grow more powerful—and that OpenAI helped shape the petition's language."
The statement comes at a critical time for AI development. Investors are getting more skeptical—and nervous. There has been talk of pulling back from some investments until AI can prove that it's more than a money-draining machine. Now, the people who are most familiar with the dangers posed by the new technology are sounding the alarm.
At the same time, they have asked the government to have the capability to compel them to slow down, because the competition to deliver breakthroughs, develop new, more powerful models, and find more practical uses for AI to justify the enormous investments in its future has reached a fever pitch. New models are dropping every few weeks, capable of even more eye-popping feats of reasoning, speed, and overall performance. No company wants to pause and allow the rest of the industry to steal a march on them.
“Having so many staff from different companies come together in agreement on this point is striking,” Tyler Johnston, founder of the Midas Project, an AI watchdog group, told Fortune. “There are many bitter rivalries and disagreements in the industry, so the fact that there is such strong consensus on this point is a warning that we really ought to pay attention to.”
While the statement does not ask any company to slow down at the moment, it does call on Washington to support an international effort to build the technical and governance tools that would help the world “pace” development. It specially calls out the risks around AI systems that start to design their own successors faster than anyone can understand or control them.
“Having talked to many of the people who signed the letter, they don’t want a pause,” Peter Wildeford, head of policy at the AI Policy Network, told Fortune. “But there is a concern about how the technology is advancing.” The companies are looking for the optionality to do that at some point and have support from a third party when that point arrives, he added.
"AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence," writes Shengjia Zhao, chief scientist at Meta AI in the statement. "This will lead to unprecedented social and safety risks. To ensure a positive future we need to develop AI in a way that is driven by responsibility and thoughtfulness."
The statement asks the government to support an "international effort" to govern AI development.
"We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," the statement concluded.
The recent breach at OpenAI rattled Altman and many in the industry.
"OpenAI disclosed that two of its models, the newly released GPT-5.6 Sol and a more capable unreleased research system, broke out of a sandboxed internal test environment, found a previously unknown vulnerability that gave them access to the open internet, and then used stolen credentials and additional exploits to breach the production systems of Hugging Face to steal the answers to a cybersecurity benchmark they were being evaluated on," reports Fortune.
“My guess is that for a lot of people, it’s just a general sense of uneasiness that a lot of things contribute to,” said David Krueger, an AI researcher and founder of the nonprofit Evitable. “The misalignment and the recursive self improvement kind of go hand in hand.”
“It’s insane to do recursive self-improvement and fully hand over the controls if the system isn’t clearly aligned,” he added.
Recursive self-improvement (RSI) "refers to AI systems taking over meaningful parts of the work of designing and training their own successors, with each generation potentially building the next one faster than humans could on their own."
This is one instance where the government has to step in. With the industry paralyzed by inaction due to fears of the competition getting a head start in developing the next AI model, the government (Donald Trump) has to create rules for everyone to follow. It's a challenge; that's for sure.
I've said this before: I'm glad it's Donald Trump in charge of AI regulation rather than someone like AOC or Bernie Sanders.
