Monday, August 10, 2020

The more you know about self-driving cars the less safe you will feel

I've been saying this for quite some time, and every so often something else comes out about how the self-driving systems are designed.  It all just reinforces how bad these are.  Another of these revelations is out and it's a doozie.

It seems that testing shows that self-driving cars are running into stopped cars, at least during test trials.  They don't have any trouble avoiding moving cars, just cars that have pulled over partly onto the shoulder.  The reason will blow your mind:

Radar has low angular resolution, so it had only a crude idea of the environment around the vehicle. What radar is quite good at, however, is figuring out how fast objects are moving. And so a key strategy for making the technology work was to ignore anything that wasn't moving. A car's radar will detect a lot of stationary objects located somewhere ahead of the car: these might be trees, parked cars, bridges, overhead signs, and so forth.

These systems were designed to work on controlled-access freeways, and, in the vast majority of cases, stationary objects near a freeway would be on the side of the road (or suspended above it) rather than directly in the car's path. Early adaptive cruise control systems simply didn't have the capability to distinguish the vast majority of objects that were near the road from the tiny minority that were on the road.

So cars were programmed to focus on maintaining a safe distance from other moving objects—cars—and to ignore stationary objects. Designers assumed it would still be the job of the human driver to pay attention to the road and intervene if there was an obstacle directly in the roadway. [emphasis added by me - Borepatch]

This is exactly why I have been saying that I don't trust these systems.  I don't know what design assumptions went into them, or into the components that make up the system.  It may even be that the system designers don't know all the assumptions, at least those in the components they use.

And the assumptions can kill you.

This article describes a test done by the American Automobile Association - well done, AAA.  There's a lot of ass covering by the auto manufacturers saying that drivers are ultimately in charge, yadda yadda.  Here's the problem with that:
This may be a fundamental problem with this approach to driver assistance technology. The ADAS is supposed to do most of the driving, but the human driver is supposed to still monitor the system and make sure it doesn't make mistakes. But our brains aren't wired for this level of monotony. Monitoring a system that works correctly 99 percent of the time is in some ways harder—not easier—than just driving the car yourself. And monitoring a system that works correctly 99.9 percent of the time is even harder, because it's that much easier for our brains to get distracted by something else.
Not to mention that this isn't what marketeers are selling, or what people want to buy.  People want "Drive me to the supermarket", not "do some of the driving to the supermarket while I do some of it, too."  That's a hard sell for a system that must add thousands of dollars to the vehicle's purchase price.

Me, I want to know how to disable these damned things.


libertyman said...

Or time to locate that older car you always wanted, before all of the computer aided everything, and hang on to it. How about that 1966 GTO you admired?

LindaG said...

I read "I, Robot" growing up.
Saw the Will Smith version.
Never wanted, and still don't, a self driving vehicle.

Joe Texan said...

Your post states clearly some of the things I've been worrying about with self-driving cars. I'll stick with my DIY (drive-it-yourself) cars and my Gold Wing. I can't imagine trying to maintain awareness with the car doing 99% of the work.

Now I'm worried about the OTHER drivers in their self-driving cars.

Ratus said...

BP, they aren't "self driving" cars they're "self crashing" cars.

They just need to get crash rate down to lower than the human one and everything will be fine.

Borepatch said...

That Goat sure is sweet, Libertyman. I wouldn't have to worry about it driving itself, but I *would* have to worry about everyone wanting to drive it ...

LindaG, yup.

Joe, that's why I'm nervous on the Harley. Drivers in Florida are idiots.

Ratus, I'm not sure about that. If the crash is their fault, lawyers will come after them. The opportunity to ask embarrassing questions about the design decisions is a target rich environment.

SiGraybeard said...

The problem of coping with systems that are hands-off for 99% of the time has been going on with commercial aviation autopilots for quite a while, and it's a recognized problem. By the time the instruments figuratively say, "I can't figure out what's going on - you take the controls" the pilot is usually so far removed from the situation that they can't catch up before the accident.

An additional problem here is there's always a technical add-on that can refine the system but still leaves doubt that they're really going to be safe. For the radar, increase angular resolution by going up in frequency or making the antenna bigger. There's a lot of talk about going up to optical frequency radar: LIDAR. That still requires software. Can they make that software safe?

Beans said...

Libertyman - the problem is, the same people who are pushing fully autonomous vehicles are also pushing the 'once we're fully autonomous in modern vehicles, you peons with old cars have to turn them in.'

California has already pushed to eliminate non-driver assist cars once the technology is semi-viable.

Like.. smart guns.

Witold Pilecki said...

I remember a discussion a few years ago about the decision making abilities of the self-driving car versus a human driver. Given the choice of running into a group of school children in the roadway or purposely running off the road into unknown dangers (like a giant oak tree) will the self-driving car sacrifice the children to preserve the life of the occupants that own it? Who would make, sell, or buy a car that might purposely kill the occupants? Those are not decisions I want made by a robot on my behalf.

As it is, every time I get behind the wheel of my wife's 2017 RAV-4, I shut off all the obnoxious lane departure, sway, and pre-collision braking systems. They don't stay off, and reset on the next startup.

Glen Filthie said...

They are still evolving.

Hey - if you can make something like the V22 Osprey fly, or have rockets that land themselves... self driving cars are a piece of cake. I'd like to see a real comparison of body counts: how many people killed by self driving cars, vs how many killed by drunks, morons, and ethnic imports.

Old NFO said...

Hrmmm... 66 GOAT, I had one, would love another one, but I'd damn sure upgrade the brakes!!! :-D

Greg said...

Our standard sendoff when anyone leaves our house is "Watch out for idiots!" These "autonomous" cars are just another kind of idiot to watch out for. It used to be just drunks and teenagers who think they're immortal. Now we have whacked out meth freaks, dopers, and the half-blind geriatric set who refuse to give it up.
I made a career out of troubleshooting robotics in medical technology and always said that 'droids are idiots who can't tell you what's wrong and I had to puzzle it out. But at least our 'droids in the lab were bolted to the floor and didn't wander around loose like R2D2. Autonomous semi-trucks on the interstate with no driver aboard? That's a lot of mass and momentum driven by an idiot.
I loved my motorcycles, but I gave it up many years ago because of the fact that everyone else on the road is TRYING to kill you.

McChuck said...

We still don't have self driving trains. That, you know, literally run on rails.

Steve said...

Are Americans so lazy that they don't want to drive their fat a** around.....they want self driving cars!??!!? Then take a bus or train!

Ritchie said...

While I would have no truck with self driving cars, Boston Dynamics uses a laser scanning system to build a map of the device's surroundings. Perhaps a better way would be to invent self-driving quadrupeds. Oh, wait.

Rick T said...

I'm on my second generation of Ford cars with AAC and the new unit is a bit better handling the 'stopped car unmasked by lane change' scenario at stop lights but it isn't perfect.

Even so, I was the sole driver on a 15 day 5,000 mile road trip recently and was able to use AAC almost all the time to help with pacing cars ahead of me and maintaining reasonable speeds.

Ken said...

It'll be "Red Barchetta" time soon.

Aesop said...

Any company unwilling to strap their CEO, boards of directors, and R&D staff, in toto, to their self-crashing cars' bumpers for the first 10,000 miles of driving should be banned by law from selling them to the public.

The whole idiotic undertaking is the zeppelin Hindenburg crossbred with RMS Titanic, on the prowl for the fortuitous conjunction of a busload of children, and a few hundred rapacious liability lawyers.

Then it's all over but the fire sale, following by a few fine public lynchings.

Callou! Callay! O frabjous day!