I listened to a guy relate his story about a simulation run by the U.S. Air Force that was the software for an AI drone being developed for said Air Force. The mission was to fly over terrain in "enemy territory" and destroy SAM sites. The human overseer was supposed to have Go/No Go on the target destruction. The drone acquired a target and was told by the human operator "No Go", at which time the drone decided the operator was an obstacle to the mission and the drone then attacked the site where the operator was located. The drone was reprogrammed NOT to destroy the operator site, and the simulation was restarted. This time, whenever the operator gave a "No Go", the drone would destroy a communications facility to prevent the operator from giving a "No Go". The simulation was stopped for evaluation. It sounds like we should read Isaac Asimov's I Robot and find a way to install the First Law of Robotics on our AI systems. (edit: I tried to edit the title to AI but it would change it it in thread tools.)
AIs are, IMO, alien intelligences. They are going to come up with ways of responding to things that will be unpredictable, even inconceivable, to us once they become advanced enough. These are just the baby steps we’re seeing now.
Yeh, I read that, too. I commented elsewhere that such things are totally foreseeable to any of us who grew up reading sci-fi; or more specifically, reading dystopian writings. In fact, I imagine it gave some number of us a chuckle. Arrogant humans actually tried it again. edit to add: Part of the issue was the Points reward system the drone got for taking out a SAM. The drone did what it was being rewarded to do, and took out all impediments (like the operator saying "Don't Kill The SAM.") The reprogramming to not kill the operator actually deducted points for doing so, thereby providing a disincentive. That's when the drone went to the next link in the impediment chain, being the comm tower.
Every Title word is Proper Case. You cannot force any other letters to be capitalized...you have to insert a space so they appear to be new words. The best you can do is to type "A I" or write out the entire words.
Now the Air Force is backtracking on the story, claiming that the story was "anecdotal," and the remarks were taken out of context. link