Former OpenAI and Anthropic researcher Jacob Coxon has issued a disturbing warning about the race to build advanced artificial intelligence. In a viral post on X, Coxon accused both companies of moving toward self-improving superintelligence without adequate safeguards, potentially putting humanity at risk.
Coxon claims that many researchers developing powerful AI systems genuinely believe the technology could cause human extinction before the end of the decade. Anthropic Alignment Science Lead Evan Hubinger publicly supported the warning and estimated the chance of such a disaster at more than 10% within the next ten years. However, he clarified that today’s AI models present a low immediate risk and that his concern involves future systems capable of repeatedly improving themselves.
The warnings have intensified debate over whether AI companies are advancing faster than governments and safety researchers can respond. Coxon has suggested stronger government oversight, cooperation among leading AI laboratories and, if necessary, a temporary pause on increasing model capabilities. OpenAI research leader Jakob Pachocki has also raised concerns about understanding and controlling increasingly capable AI systems.
These predictions remain disputed and do not prove that AI will destroy humanity. Still, warnings from researchers inside leading AI companies highlight the need for independent safety testing, enforceable regulations, transparent incident reporting and clear limits on autonomous systems before their capabilities move beyond effective human control. The real question is, what do we to stop it?