We have all seen the warning that AI can make mistakes. For a chatbot, that usually means inventing a fact, misunderstanding a question, or confidently giving you the wrong answer. Put the same technology inside a weapon, and those mistakes can end very differently.
A New York Times investigation has detailed a July 6 Russian drone attack in Zaporizhzhia, Ukraine, that killed three civilians, including a 19-year-old university student. Human operators programmed the drone to travel toward a gas station, but an autonomous AI system selected the exact object it would attack, most likely propane tanks. The drone failed to reach that target and exploded nearby.
An expert cited by The Times described it as the first documented case of civilians being killed by a Russian drone using this kind of self-targeting AI system.
A computer is now choosing what gets hit
Earlier AI-assisted drones generally kept a human involved in selecting the target, leaving software to handle the final approach. The system recovered from Zaporizhzhia went further.
Investigators found an Nvidia Jetson Orin computer onboard, while Ukrainian officials said its code revealed the types of objects the system had been trained to recognize and attack. Fully autonomous targeting removes the human from that final target-selection process.
A 1979 IBM training manual carried a warning that feels disturbingly relevant today: “A computer can never be held accountable, therefore a computer must never make a management decision.”
Nearly five decades later, computers are being trusted with decisions carrying far greater consequences.
Who takes responsibility when AI gets it wrong?
An autonomous weapon cannot stand trial, explain its reasoning, or face punishment after a wrongful death. Human responsibility does not disappear, but assigning it becomes far more complicated when software makes part of the targeting decision.
Russia is also not alone in pursuing this technology. Ukraine has tested fully autonomous systems against fuel storage sites and military equipment in Russian-occupied Crimea, although its former defense minister told The Times that those tests had not killed civilians.
Autonomous AI weapons are already being used in an active war, and civilians have now died during their use. Russia is testing them, Ukraine is testing them, and the technology will only become more capable from here. This is a dangerous slippery slope, because once militaries start handing targeting decisions to software, pulling that responsibility back may become increasingly difficult. There may be no going back, which makes meaningful limits on how these systems are used all the more urgent.