Students at MIT in the US claim they have developed an algorithm for creating 3D objects and pictures that trick image-recognition systems into severely misidentifying them. Think toy turtles labeled rifles, and baseballs as cups of coffee.This has been a problem in my field (computer security) for about as long as I can remember - certainly back into the 1980s, and almost certainly longer. Programmers work to get functionality correct - the program performs as intended when fed normal (i.e. expected) input. Programmers have done poorly anticipating what a Bad Guy might feed the program as input. Instead of a name in a text field, how about a thousand letter A characters? Oops, now the Bad Guy can run code of his choice because the program didn't anticipate this input and then fails in an uncontrolled way.
And now AI looks like it's doing precisely the same old pattern:
The problem is that although neural networks can be taught to be experts at identifying images, having to spoon-feed them millions of examples during training means they don’t generalize particularly well. They tend to be really good at identifying whatever you've shown them previously, and fail at anything in between.
Switch a few pixels here or there, or add a little noise to what is actually an image of, say, a gray tabby cat, and Google's Tensorflow-powered open-source Inception model will think it’s a bowl of guacamole. This is not a hypothetical example: it's something the MIT students, working together as an independent team dubbed LabSix, claim they have achieved.Oops.
“Our work gives an algorithm for reliably constructing targeted 3D physical-world adversarial examples, and our evaluation shows that these 3D adversarial examples work. [It] shows that adversarial examples are a real concern in practical systems,” the team said.
“A fairly direct application of 3D adversarial objects could be designing a T-shirt which lets people rob a store without raising any alarms because they’re classified as a car by the security camera,” they added.Double oops.