My Dad used to love to say that the reason that history repeats itself is that nobody listens the first time. We see this in computer security all the time, where the exact same mistakes that have been made over and over are repeated in a new technology field.
One of my favorite examples is from the early days of Internet shopping. Some of the first shopping cart software ran mostly on the browser (it was written as client-side javascript). People would select an item, then save the web page locally. Then they would use an editor to find the price in the javascript and change the price to a dollar. Reload the locally saved page, click "Buy Now" and voila - you bought a laptop computer for a dollar.
The problem is that the e-commerce system didn't validate inputs correctly. Fast forward 20 years to today and guess what self-driving car AI doesn't do?
Now, I'm not sure quite how you'd go about validating that sort of input, but it's pretty darn important data. I mean, someone could die. This is a very hard problem to solve, but it's exactly the sort of problem you'd need to solve.
Imagine a projection of a traffic sign that clogs up the Washington beltway at rush hour (I mean, more so than normal). The bottleneck would cascade as people run out of gas until nothing could move for hours and hours. And the attack looks trivial. There are likely a lot of variations that could tie everything in knots.
This is exactly why I have been so vocally skeptical about the viability of self-driving cars. The easy problem is getting the AI to work under normal circumstances ("easy" is actually pretty hard, but it certainly doable). What's hard is to make the system robust under attack. Saying "why would anyone want to attack the system" is actually a sign that whatever you are building will never be fit for purpose.
Look at all the problems that fly-by-wire has. Autonomous vehicles have all the problems of fly-by-wire while operating in an environment where obstacles are within feet of the direction of travel. Nor does this account for intentional attempts to hack the autonomous vehicle networks.
ReplyDelete"Some nut went out to the Army/Navy store, bought himself a stop sign..."
ReplyDeleteYeah, some of Bill Cosby's old material is still funny. Right up until it happens in real life.
Fine, I'll bite: what problems does fly-by-wire have?
ReplyDeleteOne doesn't need high tech to create such problems; a few decades ago my local gummint responded to complaints from 2 residents about speed on a particular street, so they solved that by moving the stop signs - that had been there for 40 years - from the cross street to the main street.
ReplyDeleteThis produced lots of accidents for the obvious reason, so after about 4 months, dozens of crashes and multiple complaints from the houses on the corners about cars in their yards and hitting their houses, they moved the signs back, producing a fresh group of accidents.
I see an opportunity here for low-cost automation of traffic crashes and traffic jams and not much else.