The falacy of stopping bad things from happening

I've been working in computer security for a long time, and have seen a lot of new technologies introduced, some to stay, some just to come and go.  All of these have been inspired by the same goal: make the bad thing go away.  Make it so it won't happen again.

One of these which I was involved with almost did this.  We almost changed the world.  Almost.  But it was too complicated, and too hard to make work in the Real World.  We didn't really understand the consequences for the people who would have to manage it when we came up with the idea, and so it's gone now.

We see this all around us.  There's a deep, primal desire to make the world intelligible.  Whether it's sacrificing virgins to the Volcano Goddess or putting one more scanner or procedure in place at the airport security check point, the motivation is the same.

Make it so it won't happen again.

Alas, I've come to despair of the new security technology that will do this.  Now I look to prevent mistakes that are preventable, which means that we have to live with the mistakes that were unpreventable.  Now I realize that like Heisenberg's Principle - so often used as an inappropriate analogy, like here - taking action to change something also has consequences.  That some of those consequences can be bad - maybe not as bad as what you're trying to prevent, but certainly worse than an alternative you might have chosen.

I've come to believe that thinking things through before you act is one of the ways that you can prevent the preventable mistakes.  Well, some of them, anyway.

Tam says this about self defense and when you should shoot, and when you should be somewhere else.  Some things are unpreventable, and we just need to adjust, and move on as best we can.   But other things are indeed preventable mistakes.

If you're wrong in computer security, people don't die.  That's one of the nice things about this branch of the security career tree, that the Bad Guys don't shoot at you.  As far as I can tell (and I've looked, quite hard over the years) nobody has ever died because of a computer security failure.  Nobody has ever made a preventable mistake in their firewall rule set, or certificate revocation mechanism that killed someone graveyard dead.  The consequences of the choices we make in my industry can end careers, but not lives.

That's one of its charms, when you think about it.

We're surrounded by choices, whether driving home from the bar after (maybe) one too many or how we interact with our fellow citizens when packing our heater of choice.  Choices have consequences, even when you choose right.

And by "right" I mean "avoid preventable mistakes".

It's not possible to stop all bad things from happening.  We should understand that we can make things worse than they would have been, if we're not careful.  You all know you should read Tam, and while it goes without saying I'll say it anyway.  RTWT.