Anthropic says it has blocked multiple users who attempted to use its artificial intelligence models for research that could potentially contribute to the development of biological weapons.
In a report released Thursday and cited by CNN, the company behind Claude described biological misuse as one of the “most serious risks" associated with increasingly capable AI systems.
Anthropic said it examined 30 days of activity and identified approximately 35 “distinct research efforts" involving potentially concerning behavior. The company stressed, however, that it could not determine with certainty whether the individuals involved intended to cause harm or were pursuing legitimate scientific research.
The report presents five case studies involving users who bypassed safeguards, including by circumventing geographic restrictions or concealing the purpose of their work. The cases included research that could involve gain-of-function techniques, bird flu and other infectious diseases, as well as novel venoms and toxins. Anthropic said the individuals were “working scientists," but did not disclose their institutions or countries.
“Sophisticated attacks no longer require sophisticated attackers," the report states. “The cybersecurity skills of AI models means that AI has collapsed the labor and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators."
One case involved a request to help prepare a grant application seeking funding for gain-of-function research into the transmissibility and immune evasion of chikungunya virus. Another involved a researcher outside the United States using Claude to study bird flu, including how the virus adapts to mammals and causes severe disease.
Other examples involved orthopoxviruses, a family that includes the viruses responsible for smallpox and mpox, as well as research involving venom toxins.
Anthropic noted that AI-assisted drug development remains one of the technology's most promising applications, with potential benefits for treating cancer and other diseases. At the same time, the company warned that capabilities useful for discovering medicines and other biological applications could also be exploited to develop biological weapons.
The company said it aims to support legitimate scientific research while recognizing that newer AI models can perform increasingly complex tasks. Anthropic warned that malicious users could exploit legitimate biological applications to maintain “plausible deniability" around potentially dangerous research.
“As our models become increasingly capable, approaching or exceeding expert performance at challenging scientific tasks, we expect their impact will only increase, both in beneficial and potentially harmful contexts," the report said. “As Al models become more widely used, providers will continue to acquire threat-relevant visibility into real-world use that even governments and intergovernmental organizations lack."
Anthropic said its newer models, including Claude Fable 5, have “stronger safeguards that restrict access to a wide range of dual-use biological research queries." The company said there is evidence its existing protections are effective, while emphasizing that stronger safety measures will be needed as both AI capabilities and potential threats evolve.
The report also said Anthropic has disrupted attempts to use its models for surveillance operations, scams and the development of conventional weapons such as drones and missiles.
The disclosure comes amid increasing warnings from former AI employees about the pace of development and the industry's ability to manage the technology's risks. Some former researchers have argued that governments may eventually need to intervene if AI development advances beyond humanity's ability to control it.
Among them is Jacob Coxon, an AI researcher who previously worked at Anthropic and OpenAI. Coxon announced this week that he was leaving Anthropic and the AI industry, citing serious concerns about the direction and speed of advanced AI development.
Coxon, who worked on AI model pretraining, said his experience at two of the world's leading AI companies had convinced him that neither was acting responsibly enough in light of the potential dangers.
“They are racing directly toward self-improving superintelligence and gambling with our lives," Coxon said, according to reports in The Wall Street Journal.
Coxon's primary concern is the possibility of AI systems becoming capable of improving themselves and advancing rapidly beyond effective human oversight. He argued that competition between leading AI laboratories makes it difficult for any individual company to slow its development even when researchers recognize serious risks.
Coxon also warned that people working directly on advanced AI believe the technology could pose an existential threat to humanity before the end of the decade. He said future superhuman systems could potentially conduct sophisticated cyberattacks, accelerate scientific and technological advances, and gain substantial influence and resources in the physical world.