Jacob Coxon, an artificial intelligence researcher who worked at Anthropic, the company behind Claude, and previously at OpenAI, has announced that he is leaving Anthropic and the AI industry, citing serious concerns over the direction and speed of advanced AI development.
Coxon, who worked on AI model pretraining, said that his experience at two of the world's leading artificial intelligence companies left him convinced that neither is acting responsibly enough in the face of the potential risks.
“They are racing directly toward self-improving superintelligence and gambling with our lives," Coxon said, according to reports in the Wall Street Journal.
His central concern is the prospect of developing AI systems capable of improving their own capabilities and rapidly advancing beyond effective human control.
Coxon argued that intense competition among the world's leading AI laboratories makes it difficult for any individual company to slow down, even when researchers inside those organizations recognize potentially serious risks.
He also drew a distinction between the cultures at OpenAI and Anthropic. According to Coxon, the risks are more widely understood within Anthropic, but the company continues pushing forward partly out of concern that a less safety-conscious competitor could reach highly advanced AI first.
That dynamic, he warned, creates a race in which even companies that emphasize AI safety remain under pressure to build increasingly capable systems.
Coxon also issued a particularly stark warning about the possible consequences of the technology. He said people directly involved in building advanced AI genuinely believe there is a possibility that the technology could pose an existential threat to humanity before the end of the decade.
He further argued that future superhuman systems could potentially carry out sophisticated cyberattacks, accelerate scientific and technological breakthroughs, and acquire significant influence and resources in the physical world.
Despite his criticism, Coxon did not claim that Anthropic is ignoring AI safety. He acknowledged that the company has invested substantial effort in the field, but questioned whether any single laboratory can responsibly develop extremely advanced artificial intelligence while competing in a rapidly accelerating commercial race.
Coxon called for stronger government rules and broader agreements among leading AI companies that could allow the industry to slow development without putting individual companies at a competitive disadvantage.
His departure adds to an ongoing debate within the technology industry over whether the rapid improvement of artificial intelligence systems is outpacing efforts to understand and control the risks they may create.