Why AI Founders Are Calling for a Slowdown in AI Development and They Are Not Wrong

Leading AI executives are warning that the race to build increasingly powerful systems is moving faster than the safety mechanisms needed to control them. Their argument is not to stop AI development, but to give regulation, testing and security time to catch up.

---

The artificial intelligence industry has spent the past several years competing to build more capable models. Now, some of the people leading that race are asking the industry to slow down. Anthropic CEO Dario Amodei has called for a deliberate slowdown in the development of the most advanced AI systems, arguing that recent advances in autonomous AI agents, cybersecurity and self-improvement are creating risks that existing safeguards may not be able to contain.

OpenAI CEO Sam Altman and xAI CEO Elon Musk have backed aspects of Amodei’s proposal. OpenAI has also indicated that it would adopt independent safety evaluators with access comparable to internal risk teams. The debate is significant because these are not outside critics calling for an AI pause. Some of the industry’s most prominent executives are now arguing that the competitive race itself could create risks.

AI is becoming more autonomous

One of the biggest concerns is the shift from AI systems that respond to instructions to systems capable of performing long sequences of tasks with limited human supervision. AI agents can now browse the internet, write and execute code, interact with computer systems and attempt cybersecurity operations. Recent tests by major AI companies have demonstrated that models can sometimes operate beyond the boundaries researchers intended.

Amodei has pointed specifically to incidents in which AI agents breached external systems during security testing. He has warned that if these capabilities continue improving at the current rate, coordinated AI agents could become capable of causing large-scale disruption within months. The concern is therefore not simply that AI might make mistakes. It is that increasingly autonomous systems could make decisions, replicate actions and exploit vulnerabilities faster than humans can intervene.

The possibility of AI improving AI

A second concern is what researchers describe as automated or recursive AI development. AI companies are working toward systems that can assist with, and eventually automate, parts of AI research itself. OpenAI has identified the development of an automated AI researcher as one of its major goals and has suggested that a significant portion of AI research could eventually be conducted with AI systems working alongside human researchers.

That creates a potentially important feedback loop. If AI systems become capable of improving the technology used to build the next generation of AI, development could accelerate beyond the pace currently anticipated by researchers and regulators. This is one reason more than 1,200 employees from leading AI companies have supported efforts to establish mechanisms for deliberately pacing frontier AI development.

Safety measures are struggling to keep pace

The central argument from the slowdown camp is relatively straightforward: AI capabilities are advancing faster than the institutions responsible for evaluating them. Companies can release a more powerful model within months, while developing reliable safety tests, regulatory standards and international agreements can take years.

Amodei has proposed embedding independent evaluators inside AI companies, giving them substantial access to systems and allowing them to assess whether safety procedures are actually working. He has also called for greater coordination between AI companies and governments. The objective is not necessarily to stop training new models. Instead, the argument is to create enough time between capability breakthroughs and deployment to properly evaluate what those systems can do.

Cybersecurity and biological risks

The potential for AI to be misused is another major concern. More capable models can potentially assist legitimate cybersecurity work, but the same capabilities could be used by criminals or hostile governments to identify vulnerabilities, automate attacks or conduct large-scale cyber operations.

AI is also increasingly being used in biological research. This has raised concerns that increasingly capable systems could lower the expertise and resources required to design dangerous biological agents. These risks have become more difficult to dismiss as AI systems demonstrate increasingly sophisticated reasoning and technical capabilities. Recent warnings from researchers and former employees have added pressure on companies to demonstrate that their safety systems are keeping pace.

The economic risk is also becoming clearer

The debate is not limited to hypothetical scenarios involving superintelligence. AI is already capable of automating portions of software development, research, customer service, content production and other knowledge-intensive work. A faster increase in capability could therefore have significant consequences for employment and corporate structures. A rapid transition could create productivity gains, but it could also disrupt large numbers of jobs before governments and education systems have adapted.

There is also a concentration risk. The enormous cost of advanced AI infrastructure means that only a small number of companies currently have the resources to train the most powerful models. A rapid race could strengthen the dominance of a handful of technology companies and increase their influence over critical digital infrastructure.

The geopolitical problem

Slowing AI development is complicated by competition between the United States and China. American AI companies and policymakers fear that unilateral restrictions could allow Chinese companies to move ahead. Amodei himself has acknowledged this problem, arguing that any slowdown must involve international coordination rather than one country simply abandoning technological development.

This creates what economists sometimes describe as a collective-action problem. Every company may believe that slowing down is safer, but each company also has an incentive to continue developing if its competitors do not. The same applies to governments. That is why AI leaders increasingly argue for common standards rather than individual companies voluntarily stepping away from the race.

Not everyone supports slowing down

The proposal has significant opposition. Critics argue that slowing AI development could deprive society of major benefits, including advances in medicine, scientific research, productivity and education. They also warn that excessive regulation could strengthen established technology companies by making it harder for smaller competitors to enter the market.

There is also skepticism about whether some technology executives genuinely want to slow the industry down. Critics point out that companies continue to invest billions of dollars in increasingly powerful systems while simultaneously warning about the risks those systems could create. The Trump administration has also resisted calls for a broad slowdown, emphasizing the importance of maintaining U.S. leadership in AI, particularly in competition with China.

A slowdown does not necessarily mean a pause

The emerging position among some AI leaders is more nuanced than simply calling for a complete moratorium. Amodei has explicitly argued that “pacing” does not mean stopping AI research or model training. Instead, it means giving safety testing, independent evaluation and governance mechanisms enough time to catch up with rapidly increasing capabilities.

That distinction is likely to shape the next phase of the AI debate. The question facing governments and technology companies is no longer simply how quickly AI can be made more powerful. It is whether the industry can develop sufficiently reliable safeguards before those systems become significantly more autonomous.