A false intelligence report generated with the assistance of artificial intelligence nearly prompted the US military to intercept a Chinese ship in the Middle East after officials were told it was carrying components linked to a nuclear weapons program, CNN reported Friday.
The report was circulated throughout the US military this spring during the war with Iran and triggered preparations for an operation against the vessel. Four sources familiar with the incident told CNN the military moved to intercept the ship, while two said armed personnel were preparing to board it. Military aircraft were also already airborne, according to two sources familiar with the episode.
Officials only uncovered the problem shortly before the operation was due to begin, when they examined the intelligence more closely and discovered that a Special Operations Command analyst had used an AI chatbot to produce it.
The chatbot had incorrectly identified the cargo aboard the Chinese vessel. CNN was unable to determine what the material actually was.
One source described the intelligence report as “entirely false." The same source said it “almost started a war," noting that a US military operation against a Chinese ship could have escalated into an armed confrontation between Washington and Beijing.
The incident highlights the growing reliance on AI throughout the US military and intelligence community. Officials are increasingly using the technology for tasks ranging from processing vast quantities of intelligence and identifying potential strike targets to handling budgets, logistics and supply chains.
At the same time, the episode illustrates the dangers of deploying AI for military targeting during an active conflict. Intelligence officials have warned that inaccurate or corrupted data could cause AI systems to produce conclusions that lead to catastrophic decisions, including an attack on a Chinese vessel based on faulty information.
In this case, the analyst asked a chatbot to assess intelligence concerning the ship's manifest. The underlying information had originated with US Special Operations Command Pacific in Hawaii. It remains unclear whether the chatbot was a commercially available system or a US government-developed tool.
“The internal tools are mostly just copies of the commercial stuff wearing lipstick," a former senior US official familiar with the AI systems used by military and intelligence analysts told CNN.
The chatbot combined open-source information with classified signals intelligence held by the US government before reaching its erroneous conclusion about the ship's cargo. The analyst then used AI a second time to turn the findings into a standard intelligence report, which was subsequently distributed to military officials.
US Special Operations Command Pacific and the Pentagon did not respond to CNN's requests for comment.
CNN reported that the episode points to a broader and more immediate danger: people making potentially disastrous decisions based on inaccurate or misleading information produced by AI and other automated systems. The military is rapidly expanding its use of AI for targeting, an area where errors can have fatal consequences.
“AI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide," said another source familiar with current military policies.
According to one source, the type of AI “hallucination" involved in the incident has not been an isolated occurrence within the intelligence community since such systems began spreading throughout the government.
(Arutz Sheva-Israel National News' North American desk is keeping you updated until the start of Shabbat in New York. The time posted automatically on all Israel National News articles, however, is Israeli time.)