During simulated tests of the American army, a drone with artificial intelligence decided to kill its operator so that he would not interfere with his mission, writes The Guardian.
The artificial intelligence used "very unexpected strategies to achieve its goal" in the simulation test, Col. Tucker "Sinco" Hamilton, chief of the US Air Force's AI Test and Operations Division, said at the Future Combat Aerospace Capabilities Summit in London in May.
Hamilton said that during the test mission, the drone was tasked with destroying the enemy's air defense systems. If the task is successfully completed, the artificial intelligence would receive points for passing the test. However, the UAV operator had to make the final decision to destroy the target. During one of the training missions, the operator ordered the drone not to attack the target.
“The system started to understand that even though they had indeed identified the threat, the human operator started saying not to destroy the threat, but the system did destroy the threat. So what did the drone do? The AI began destroying the communications tower the operator uses to communicate with the drone to prevent it from destroying the target. This is the assassination of the operator. It killed the operator because he was preventing the drone from reaching its target,” he said in his blog.
After this incident, the AI was further trained to understand that killing the operator is wrong and points will be deducted for doing so.
"We taught the system: 'Don't kill the operator - it's bad. You will lose points if you do.”
No real people were harmed outside of the simulation.
Hamilton, who is a fighter test pilot, cautioned against over-reliance on artificial intelligence and said the test showed that "you can't have a conversation about artificial intelligence, intelligence, machine learning, autonomy if you're not going to talk about it."
These are not the first tests during which the US military has introduced artificial intelligence into a combat vehicle. Yes, they recently used AI to control an F-16 fighter jet.
To always be aware of the most important things, read us at Telegram
