Rosenband added that four-way junctions with no lights are still a nightmare for the robot cars. An example junction is California and Powell in San Francisco, which has the added bonus of two cable car lines going through it. Human motorists rely on eye contact to know when it's safe to go or just take the initiative and move first. A driver-less car gets stuck trying to safely nudge its way across the box.
This is a great overview of the problems of computer/sensor recognition of what is trivially easy for humans. There are great examples here of the problems that we overcome instantly and naturally, but which flummox the computer: the red balloon next to a green traffic light, the traffic light partially obscured by a bus, a traffic light with the setting sun right behind it which blinds the sensor."At four-way stops, oftentimes cars arrive sorta at the same time and it's a coin flip for who goes first. We have to make it comfortable for the person in the car; you don’t want the vehicle to inch forward and then slam the brakes, and you also want to be courteous to other drivers," Rosenband explained.
We handle this via common sense, but you can't program common sense. They're trying, though:
You can teach a computer what an under-construction sign looks like so that when it sees one, it knows to drive around someone digging a hole in the road. But what happens when there is no sign, and one of the workers is directing traffic with their hands? What happens when a cop waves the car on to continue, or to slow down and stop? You'll have to train the car for that scenario.
And this is the heart of the problem: you have to define literally every possible failure condition and program those into the software. Even with machine learning, there are too many to be practical. If you miss one and a car kills someone, the lawsuits will be enormous.What happens when the computer sees a ball bouncing across a street – will it anticipate a child suddenly stepping out of nowhere and chasing after their toy into oncoming traffic? Only if you teach it.
This is an outstanding article on the complexity that technologists are trying to bite off. While unstated, you get a real feel for how they want to fly high - perhaps so close to the sun that their wings will melt.