Lies, damned lies, and statistics

I don't have a problem with scientists being wrong. This happens rather a lot - a wag once described the Scientific Method as not being the process of replacing a falsehood with a truth, but rather replacing a falsehood with a more subtle falsehood.

So scientists are wrong, all the time, for all sorts of reasons. As grown-ups, we need to set our expectations accordingly.

Sometimes scientists are wrong because they lie. They do this for all sorts of reasons, but it boils down to fame, fortune, or political gain. As grown-ups, we need to once again set our expectations accordingly.

Your expectations should be permanently set to "show me your data," because this way you can catch both honest and dishonest mistakes. Science is, after all, about reproducibility. Your theory can be beautiful, but if nobody else can get your results, all you have is Cold Fusion*.

Me, I get really suspicious when someone won't publish the data. I get double-plus extra crazy suspicious when the data-gone-AWOL result is a major media story, and my suspiciousmeter gets pegged when this major story has an obvious political agenda.

Both my regular readers have seen me post about how the Global Warming scare is junk science, and almost certainly politically motivated. This bothers me - actually makes me angry - not just because someone's trying to sell me a bill of goods, but because they think I'm dumb enough that they can sell me.

Data, dude. Or later, dude.

Sometimes the scientific community takes notice, and the results are simply glorious. You may recall that back in 2004 - surprisingly, right before the election - the medical journal The Lancet published a bombshell report on civilian deaths in the Iraq war. The report claimed many, many more civilian deaths than anyone else had counted, and laid the blame squarely on the Bush administration.

My Suspiciousmeter was pegged, and I wasn't the only one:

First, even without reading the study, alarm bells should go off. The study purports to show civilian casualties 5 to 6 times higher than any other reputable source. Most other sources put total combined civilian and military deaths from all causes at between 15,000 to 20,000. The Lancet study is a degree of magnitude higher. Why the difference?

And guess what? The authors wouldn't release the data. Imagine that. Just like Mann and his global warming "Hockey Stick."

Fortunately, science is not completely politicized, and real, you know, scientists did what scientists do, and looked at the methodology:

David Kane, Institute Fellow at the Institute for Quantitative Social Science at Harvard University has authored a paper, soon to be presented, that demonstrates using detailed statistics just how deceptive (my adjective) the original study was.

Well, the wheels of science grind slowly but exceedingly fine. The author of The Lancet study has just been censured:

Gilbert Burnham, the lead author of two papers published in The Lancet on civilian casualties in Iraq, has been censured by the American Association for Public Opinion Research for failing to provide adequate details about the survey.

Not for being wrong. For hiding the data.

McArdle is pretty careful in how she presents the story:

Just to be clear: I have no reason to think that Burnham or any of the Hopkins team committed knowing fraud, as some have alleged. I don't know that there is anything wrong with their data. But the secrecy seems bizarre and wholly unnecessary, which makes it harder to trust their results.

Quite frankly, I don't agree - I vote for "fraud" here. Because they won't fully disclose their data and methodology. Reader Greg Q in McArdle's comments hammers this point home:

I have no reason to think that Burnham or any of the Hopkins team committed knowing fraud, as some have alleged.

Wrong. You have every reason to think they committed knowing fraud. Their refusal to release the information tells you that their "research" was crap, and that they know it.

Releasing the information, and having it prove that they were in error, would show an absence of "knowing" fraud. In science, the only reason to refuse to release your information is because you've engaged in knowing fraud.

Wasn't this fraud "peer reviewed"? Everyone involved in its release should be tracked down, and held to account.

Peer review is nothing (well,OK, it's something, but not a lot). What is something is reproduceability.

Let me say again, that I don't care if they're wrong, because scientists are wrong all the time. I sure as heck mind - a lot - when lousy science gets used as a club to beat political opponents.

* UPDATE 7 February 2009 10:00: Jed Rothwell points out in a comment that Cold Fusion, despite its controversal introduction, is a heavily studied field, with thousands of published papers. Real data.