Monday, April 8, 2019

The only thing dumber than a self-driving car is a self-driving tank

Peter has a post discussing what size gun for small armored vehicles, which has a lot of interesting stuff if that's your bag, Baby.  But he has a very interesting question:
In future warfare, as far as front-line combat is concerned, does infantry still have a role on the battlefield?  Is combat going to develop into a slugging match between vehicles, and possibly between unmanned systems or artificial-intelligence autonomous weapons systems?
No.

Nobody is going to put much faith into autonomous weapons systems for a long, long time.  The reason is that failure modes are much more complex than for autonomous automobiles, and are susceptible to enemy subversion.  We're actually already seeing some of this for self-driving cars, where security researchers were successful in tricking a Tesla to move into the oncoming traffic lane by putting some stickers on the road surface.  Srlsy.

So far self-driving cars have been learning (mostly successfully) to avoid obstacles on well defined roadways.  Results have been decently impressive although nowhere near good enough for me to trust my life to one of these things.  I've posted about many of the failures here, and this really boils down to a case of underestimating how difficult the problem is combined with a generous dose of Gee-Wizz marketing.  Essentially this is a problem space where rapid progress is made until the solution is 80% complete, at which point the people working the problem realize that they're facing the next 80%.

And remember, this is for driving on well marked roads with lanes painted on the surface and signposts to give a lot of clues about what's coming next.  No imagine a vehicle that has to navigate off-road, avoid obstacles, avoid damaging property owned by friendlies, all while searching for and identifying potential targets.

Remember, the targets will be actively trying to trick the vehicle's sensors and AI algorithms.  As they say, this will be a target rich environment.  I predict that the first time that a Red Team takes on one of these vehicles it will all be over very quickly.  The AI needs to do a lot more than identify obstacles on a well defined roadway, it needs to do off-road navigation while figuring out whether it is being tricked or not.

The situation is very similar to the difference between getting a web site up and running, and getting one running that is hard to hack.  The first case is just getting functionality to work as designed, the second involves ensuring that the functionality cannot be bent by clever stratagem to do something that the designer doesn't want done.

Good luck with that - this is an entirely new field, with entirely new compromise possibilities.  Head-Smashed-In Buffalo Jump is a site where plains indians hunted bison by tricking them and driving them off a cliff.  Bison are about a billion times smarter than even the best AI, and this was a viable hunting strategy nonetheless.  Is it possible to confuse a self-driving tank to drive off a cliff?  I for one wouldn't bet big money that you couldn't.

This is not a self-driving Wikipedia

And so back to Peter's question.  Yes, infantry has a place on the battlefield of tomorrow.  Quite frankly one of their uses might be to override a confused AI that is about to drive over a cliff.  Infantry will be smarter than tanks for a long, long time.

UPDATE 10 APRIL 2019 17:23: Lawrence has a very interesting take on this.  I think he's right.

9 comments:

Aesop said...

"Now imagine a vehicle that has to navigate off-road, avoid obstacles, avoid damaging property owned by friendlies, all while searching for and identifying potential targets while the enemy is shooting at it and trying to blow hell out of it.

FIFY.

Jerry said...

Old news actually. Read Keith Laumer's Bolo series. Another series having a profound impact on the defense industry is David Drake's Hammer's Slammers.

Borepatch said...

Aesop, I actually think that a preferred way to attack the vehicle would be to use a number of previously identified tricks to fool it, rather than shoot at it. If you can get it to drive itself to a location where it gets bogged down and cannot extricate itself then you've effectively disabled it.

This is speculation, but this may be pretty easy to do.

Ed Bonderenka said...

I can better imagine drone tanks than autonomous.
But infantry will always go where tanks can't.
Or won't.

ProudHillbilly said...

Worked with computers for 35 years. Was horrified but not surprised by the recent plane crashes that happened due to a software glitch.

I drive. Forget the reet.

Borepatch said...

Ed, that's a great summary. I can't argue at all.

ProudHillbilly, yup. And plane crashes are not due to someone messing with the software.

Jester said...

I can second we will see drone controlled tanks before automated ones just on a technology stand point itself, let alone human drivers/pilots are better able to respond to environmental problems like ditches, barriers and other things. It is concerning to the point of when AI decides the barriers or fire coming in is more than it can stand, will it program itself to suicide itself at that point or would it turn to fire back at those supposedly controlling it? I do think we will start to see drone type tanks on the ground in support with air drones to better see the battlefield but ground is much more reactionary to stimuli than air due to the fact there is not a lot of obstructions in the air, vs on the ground.

Lawrence Person said...

I think we're going to see autonomous drone swarms used in combat a lot sooner than you think. Soldiers are expensive. Drones are cheap and getting cheaper. Insert a drone delivery system behind enemy lines and let them wreck havoc in bases and depots. It doesn't matter if they enemy can hack one or two if you're deploying a hundred at a time at a cost comparable to one M829 round. Make some exploding, some small caliber anti-personnel, some with top-down anti-vehicle shaped charges. They don't have to be perfect, they only have to be more discriminating than artillery barrages or cluster bombs. Give them a one-time encrypted activation code and some 30-60 minutes of juice. Make them low cost enough and no one cares if you lose them all; picking up depleted drones to be sent back and reworked after you take the objective is just gravy.

If cheap enough, a drone swarm can still be plenty dumb and plenty cheap and still be lethal.

bruce said...

Lawrence, I think you have it right, the days of throwing personal at a target are over. People are reliving the past wars when they think there is a never-ending pile of bodies to throw at a fortress.
And, my little tesla 3 drives me around town and country with a little intervention.
The state of control being in beta, it still is accurate enough that I can take in the scenery with extended focus.
I'm thinking some of the commenters here are correct in their distrust of the engineering, but their inability to accept innovation and new frontiers is a failing that all of us with a bit of experience fall into. As in everything, it's a matter of degree.