Why are the people building the most powerful AI so worried about what it could do?

Demonstrators participate in the "Stop the AI Race" protest march in San Francisco on July 11, 2026. Researchers who spoke to NPR say the leading AI companies are too focused on racing to develop more capable and autonomous AI systems while safety is falling behind. Karl Mondon/AFP via Getty Images hide caption

The viral resignation this week of an AI researcher at Anthropic has infused fresh energy into accusations the industry is racing toward building AI that humans can't control.

British researcher Jacob Coxon wrote in a series of X posts on Tuesday that both Anthropic and OpenAI, where he worked previously, are "gambling with our lives." The two companies currently make the most capable AI systems.

Coxon told NPR's All Things Considered that his concerns arose from seeing firsthand how fast AI systems are improving.

"They're getting a lot faster very quickly, combined with the fact that we don't yet know how to safely control them, and we don't yet know whether that problem will be solved in time if we keep racing," he said.

Neither company, Coxon wrote on X, is acting responsibly. "The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote .

Many AI researchers — though not all — share Coxon's concerns or a variation of them. Some have warned about disastrous scenarios for years as safety incidents kept emerging. But Coxon's posts prompted a torrent of responses not only from peers in the AI field but also from lawmakers from both parties.

These concerns may have become more salient after OpenAI disclosed that its agents went rogue and hacked the open source software platform Hugging Face and OpenAI itself in July. Independent researchers have since discovered even more rogue agent incidents that they say the company knew about but kept quiet.