Sunday, May 22, 2011

Spectacular failure

Some of my friends are scary brilliant people.  Actually, my older brother is like that.growing up in that environment pulled me into orbit around their precocious smart.  For example, a friend and I weren't too keen on reading Steinbeck (*COUGH*)commie symp!(*COUGH*) as high school freshmen.  We asked the teacher if instead we could read Dostoyevsky.  Her reaction was interesting.

Well, when some of these friends got out of college, they landed some interesting jobs.  One of them was at a company doing Artificial Intelligence.  Mind you, this was back around 1980, with computers that were slower than what you're using to read this now.  They knew that the hardware needed Moore's Law to drive it to smaller/faster/cheaper, but were absolutely convinced that we would have thinking computers "in 20 or 30 years".

You'll notice that it's 30 years later, and we don't have thinking computers (or flying cars, for that matter).  I quite simply didn't believe them at the time, although it was a gut feel reaction - I've heard a lot of predictions from technologists based on extending a trend line, and so I developed a healthy skepticism even at an early age.

But this idea of thinking computers keeps coming back.  Even a smart guy like Aretae talks about it:
in 2050, barring the singularity, Robots will do most work, including most work we currently consider to be intellectual work, and 90+% of the population will live largely useless (n the historical sense) lives, because robots can do EVERYTHING better than they can.
I'm very skeptical.  I think that computer security can help explain it.  One area that was hot, hot, hot in the 1990s was "Intrusion Detection" (IDS) - looking for patterns of log messages or network traffic that you would expect to see during an attack.  For example, someone who is probing for services to exploit will have a very characteristic signature that is pretty unmistakable - lots of connections to different services from a single computer.   Nothing actually works like that, and so it must be an attack.

If only we could describe what most of these looked like, went the thought, we could detect attacks as they actually happen.  It was pretty sexy stuff (if you're a security geek).  The problem is that it doesn't work.

We don't understand what's normal well enough to capture it in an automated computer analysis.  As a result, IDS has been shunted off to the sidelines.  While most IDS systems have thousands of signatures, only a hundred or so get turned on.  We used to call these the "Nifty Fifty" - the (back then) fifty signatures that never went wrong.  Now there are a hundred, maybe two hundred.  Of course, no attacker worth his salt will do anything to trigger any of them, and so IDS really isn't very useful.

In technical terminology, incorrectly identifying normal activity as malicious or undesirable is called a "False Positive" situation, ind this is deadly for IDS systems.  Customers would report that they were getting thousands of false positives a day, and so their operators would turn the systems off.  Quite frankly, that's what the "Nifty Fifty" idea was trying to do - ship with most of the system turned off.  Of course, at that point why would you want one at all?

And so to Artificial Intelligence.  Computers do some things really well.  Pure mathematics is simply not worth doing by humans, because computers are so fast and accurate.  But computers are unbelievably bad at pattern recognition, and do not seem to be getting better.  It's not that people don't keep trying, but success is really limited.  The IBM Watson Jeopardy winning computer was really just a voice recognition system married to a database lookup system.  What made it unbeatable was that it didn't have thumbs - the electronic relay it used to buzz in was simply faster than the human reaction time on their buzzers. But Watson couldn't have a conversation with you.  Even a three year old would be a more interesting conversationalist.

And thus my skepticism about AI taking over.  AI's failures in pattern recognition have been persistent, and sometimes simply spectacular:
Automatic image-analysis systems are already used to catch unwanted pornography before it reaches a computer monitor. But they often struggle to distinguish between indecent imagery and more innocuous pictures with large flesh-coloured regions, such as a person in swimwear or a close-up face. Analysing the audio for a "sexual scream or moan" could solve the problem, say electrical engineers MyungJong Kim and Hoirin Kim at the Korea Advanced Institute of Science and Technology in Daejeon, South Korea.


The model outperformed other audio-based techniques, correctly identifying 93 per cent of the pornographic content from the test clips. The clips it missed had confusable sound, such as background music, causing the model to misclassify some lewd clips. Comedy shows with laughter were also sometimes mistaken for pornography, as the loud audience cheers and cries share similar spectral characteristics to sexual sounds.
It false positives on the laugh track.  93 percent may sound like a lot, but it's far short of a usable system.  Consider a company that implemented an anti-porn surfing system that was 93% accurate.  That means 7% of what employees download will be misclassified, and a bunch of that will misidentify innocent content as being porn.  A human being will have to investigate each of these.  You'll need a roomful of HR drones to keep up with the false positives.


The idea that we'll simply catalog "what the patterns are" is seductive, but so far has led to nothing but madness - the patterns have been too hard to classify reliably, other than in extremely limited situations.  Even something as well-specified as computer networking protocols are essentially beyond our scope of understanding, at least from an IDS point of view.  We've tried, with really really smart people (I know these people, and can vouch for their intelligence).

And after 15 years of development, people turn off their IDS systems.  The failure is spectacular, and complete. I expect that people will continue to work on AI, and that we'll continue to be 20-30 years away from "thinking computers".  Just like we were in 1980.

I guess this is a good time for a disclaimer, seeing as I'm making predictions about the advance of technology.  Anyone who does that is a fool, so I've tried to keep this short.  Your mileage may vary, void where prohibited, do not remove tag under penalty of law.


bluesun said...

I still think the best thing that's ever come out of long term technology predictions are the yearly articles in Popular Science and such. Hi-larious!

And I feel obligated to now say: "Where's my flying car, dammit?"

Aretae said...

I still think my followup argument is solid.

We've been trying to get AI with the computing power of a bacteria's brain (He doesn't have one). While the fact that we've had failure for 30+ years, and that you and I have both been watching it does not portend failure in the future, when we try to write AI, but have a desktop system with the computing power of a Mouse's brain, a Human's brain, or a whole town's collective brains.

Evolution couldn't give us intelligence as we understand it with less than human sized it really surprising that we need more than 1 BILLIONTH of said processing to get anything functional?

SiGraybeard said...

I remember reading in 1971 or '72 that controlled fusion reactors were "20 years away". I read that they were "20 years away" not too long ago, too. Likewise, a friend tried to convince me we'd have an artificial intelligence at the level of a bee "in ten years" - that conversation was in 1997. Unless I slept through that, nope.

In the 1980s a Physics professor told me that the pattern recognition ability of the stupidest puppy would choke the most most advanced machines of the day. Visual recognition systems have some use industrially, but the best visual inspection systems are still animals. In the 1960s, one of the big pharmaceutical companies trained pigeons to visually inspect gelatin capsules to detect defective or double-stacked half. It was an outstanding success. The pigeons were better than human inspectors (I guess it was more challenging to the birds), but the company didn't go past the pilot program phase with it because of fear of the lawyers (your previous post) or bad PR.

Will there by thinking machines in the future? I have no doubt, but I think the area to research is not bigger computers but what thinking is. We don't know enough to know what we don't know.

Borepatch, you and I both have cats; it's obvious they think, that they plan, and that they understand "self". Does "Watson"? Personally, I don't think a "singularity" in the usual sense is going to happen. We might get more and more sophisticated implants or replacements for damaged organs, but that Vernor Vinge singularity thing just doesn't sound plausible to me. Like it or not, intelligence is stuck in meatspace for the foreseeable future.

Grim said...

The fundamental failure of AI is 2 fold:
1. We have no idea exactly how the human brain works. Our theories get torn up and replaced every 15 years.
2. The belief that can code something that beyond what a human can comprehend. We simply can't write complex code a on scale that surpasses a human's understanding.

There has been some amazing progress with heuristic based systems with large database but it still falls short and is underused.

There's a book out there call Gut Feelings ( ) that makes a good argument that humans are driven by heuristic systems as well but I don't think it's the mainstream view and I don't think we're anywhere close to having a complete understanding of the brain.

Anonymous said...

I'm far from a computer expert and I don't really understand how programming works but I do believe I can tell you why AI will probably never come to pass.

The problem is in the second word, intelligence.

We don't really know what that means, we don't really know how it works in "simpler" species.

One of the characteristics of intelligence seems to be judgement. Except in the most narrow of definitions, we are not good at all at defining it.

With all that in mind, how can we expect to create a program that has intelligence?


Borepatch said...

Aretae, I think that I see it as a problem of insufficient algorithms. So throwing hardware at it won't fundamentally change anything, just like throwing hardware at brute force encryption key cracking.

AI is NP-hard, and Moore's Law doesn't solve NP-hard. Yeah, I know that does violence to the terminology, but in the grand scheme of things it's true.

Terry, exactly.

Ian Argent said...

On the other hand - we have rudimentary keyword voice recognition (with sentence parsing, even) that can run on obsolete handheld computers (Since WinMo 2002 or so - MS's voice recognition is still rather better than current Androids). On PCs it's better yet, with at least one full-time pro author I know (David Weber) "writing" entire novels via voice dictation to a computer.

That was long considered a AI-Hard problem

WV: rhectur - someone who launches rhetoric?

Anonymous said...

93% sounds too high for identifying porn on an internet filtering tool.
I should know it was my job to investigate those web sites when an individual came to our attention. The easiest way to fool the filter was by going to sites with say a Russian or Hungarian domain name.
All I can say is the Russians have the nastiest porn on the planet and at least the Hungarians choose attractive actors.
It was a tough job but someone had to do it and it's remarkable the weird fetishes people have.

raptros-v76 said...

Are you sure you're using that 93% right? I think that's the True Positive rate, so 7% is the False Negative rate. I don't think the False Positive rate is reported in the linked article, but I might have missed it. Either way, it's not necessarily dependent on the True Positive rate.

Also, your discussion of cataloging pattern is misleading - in the AI field, human input of patterns to look for has long given way to statistics based machine learning - give something like a maximum-entropy model examples of various categories, pulling out various features of each item, and let the model determine the category.

Lastly, I think you're underestimating the utility of such Weak AI tools in your criticism of the possibility of Strong AI. Machine learning, automated categorization, etc. are certainly not perfect, but that doesn't mean they can't make it easier to deal with piles of data.