AI Researcher Quits & Says AI Could Threaten Humanity

Jacob Coxon left Anthropic over AI safety fears as the researcher warned the global technology race could eventually threaten humanity.

The 27-year-old researcher resigned from Anthropic after working on AI pretraining at both Anthropic (Claude) and OpenAI (ChatGPT) during the past three years. His has departure put yet another spotlight on a growing conflict inside the artificial intelligence business: The same companies racing to make machines smarter are also employing researchers who fear where that competition could lead.

Coxon took his scary concerns public Tuesday night through a series of posts on X. He accused major AI companies of “racing straight to self-improving superintelligence and gambling with our lives.”

Here is the whole statement:

His argument centers on future systems rather than the AI products people currently use. OK, well that’s good to know.

Coxon believes increasingly powerful models could eventually develop abilities capable of hacking sophisticated systems while gaining access to significant resources and influence.

READ ALSO: OpenAI Is Giving Terminator Vibes

His most alarming assertion concerned what researchers allegedly say behind closed doors.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” he said.

That claim received support from some researchers in the field although it does not establish a consensus across the industry.

Evan Hubinger, an Alignment Science Lead at Anthropic, publicly backed Coxon’s characterization that some researchers genuinely consider advanced AI an existential danger. Hubinger said he believes there is a greater than 10% possibility that AI could cause human extinction within the next decade. He also stressed that today’s models represent comparatively low risk. He said this to Newsweek.

Extinction?

Coxon reportedly worked on GPT-4o while serving on OpenAI’s technical staff from 2023 to 2026. He later moved to Anthropic earlier this year where he helped train the company’s AI models. Anthropic has built much of its public reputation around research into AI safety and alignment. And that makes this news more disturbing.

His exit follows previous departures involving prominent researchers wrestling with similar questions.

Jan Leike resigned from OpenAI in 2024 after co-leading its Superalignment team. Leike publicly argued that safety priorities were losing ground inside the company.

“Over the past years, safety culture and processes have taken a backseat to shiny products,” Leike wrote.

Former OpenAI chief scientist Ilya Sutskever also departed in 2024 before launching Safe Superintelligence Inc. Unlike Leike, Sutskever did not publicly accuse OpenAI of neglecting safety when announcing his departure.

Anthropic and OpenAI both maintain formal systems intended to evaluate increasingly powerful models for dangerous capabilities. Anthropic uses its Responsible Scaling Policy while OpenAI operates its Preparedness Framework.

Those safeguards make Coxon’s departure particularly provocative. His warning is not that AI companies have ignored safety altogether. It raises the darker question of whether their protections can keep pace if machines eventually become smart enough to help redesign themselves.

For an industry selling humanity on increasingly intelligent machines, Coxon’s resignation leaves an uncomfortable possibility hanging over the race: The people building tomorrow’s technology are still debating whether tomorrow’s technology can ultimately be controlled.

And we’re watching battle rappers diss robots in the street.