Anthropic Researcher Jacob Coxon Quits AI Industry Warning Superintelligence Could Spiral Out of Control

Jacob Coxon, 27, resigned from his role as a pre-training researcher at Anthropic and announced his total departure from the artificial intelligence sector. The former OpenAI employee believes that a competitive race between top labs and Chinese rivals makes safety trade-offs inevitable in the development of self-improving systems. Coxon stated that AI capabilities are advancing at a pace where systems could spiral out of control by the end of next year.

Researchers within the industry are increasingly using terms like "crunchtime" and "endgame" to describe the speed of advancement, Coxon said. He argued that the current rush to build artificial general intelligence lacks the coordination and security of the Manhattan Project, as critical work often takes place on personal laptops in San Francisco rather than in secured bunkers.

Anthropic has marketed its safety-first approach while preparing for a potential IPO and more capable model releases. These internal warnings align with views from the lab's alignment science lead, who thinks AI has a greater than 10% chance of killing all humans within 10 years. Coxon called for government intervention and pacing agreements to slow the i