The Science Story of the Century?

I've spent quite some time looking over the chatter about the Hadley Centre's data disclosure, and have a high-level overview of what seems likely so far. While I've downloaded the data, I haven't gone through it yet. I plan to over the weekend, but this is all background on what is a terrifically interesting science story.

What is the Hadley Centre and what happened to it?

The University of East Anglia (UK) hosts the Hadley Centre for Climate Prediction and Research, one of the three most influential climate research organizations in the UK. The Hadley Centre is part of the UK Met (Meteorological) Office, the UK's national weather office. Hadley develops computer climate models and provides one of the most influential temperature data sets (CRUTEM3). Earlier this year, the Hadley Centre controversially refused a Freedom Of Information Act (FOIA) request for the CRUTEM3 raw (uncorrected) data.

Phil Jones is the current director of the Hadley Centre.

Last night someone posted 61 MB of emails, computer program code, and climate data from Hadley servers to an FTP server on the Internet. While the data has been removed from the original FTP server, it has been replicated to a number of filesharing networks, including Bittorrent. Antivirus messages that the master ZIP archive contains the virus Win32.Agent.wsg are a false positive (some antivirus scanners look for how files are compressed as a sign of being malware, and are a rich source of false alarms; that's the case here).

Hadley has confirmed the data breach.

Did they get hacked?

Nobody knows. Based on many years in Internet security, I am skeptical that this was the work of outsiders. This is speculation, but the following seems significant:

1. Email was exposed as a series of 1073 discrete files. If an external attacker were after email, he would target the email server, which keeps emails in a very small number of files. You'd have a big blob containing many emails, rather than many small blobs containing a few (or only a single) email.

2. The information looks very damaging to Dr. Jones and proponents of Anthropogenic Global Warming (more on this in a bit). An external attacker will generally not know where the bodies are buried, so to speak. A disgruntled insider, on the other hand, will have much more knowledge of where the most significant data is to be found.

What is in the information that has been disclosed?

There are lots of emails, ranging from scientific discussions to catty name calling to what may constitute a conspiracy to avoid the FOIA. Here is a sample of the most interesting emails:

On potential data manipulation (of a very warm period that makes the AGW hypothesis less compelling):
(file 1257874826.txt)

Phil Jones writes:
“One final thing – don’t worry too much about the 1940-60 period, as I think we’ll be changing the SSTs there for 1945-60 and with more digitized data for 1940-45.”

And the sausage being made here:

At 06:25 28/09/2009, Tom Wigley wrote: Phil, Here are some speculations on correcting SSTs to partly explain the 1940s warming blip. If you look at the attached plot you will see that the land also shows the 1940s blip (as I’m sure you know). So, if we could reduce the ocean blip by, say, 0.15 degC, then this would be significant for the global mean — but we’d still have to explain the land blip. I’ve chosen 0.15 here deliberately. This still leaves an ocean blip, and i think one needs to have some form of ocean blip to explain the land blip (via either some common forcing, or ocean forcing land, or vice versa, or all of these). When you look at other blips, the land blips are 1.5 to 2 times (roughly) the ocean blips — higher sensitivity plus thermal inertia effects. My 0.15 adjustment leaves things consistent with this, so you can see where I am coming from.

That 0.15 degree adjustment to the data was not just pulled out of the air; rather, it was carefully chosen to get the "right" results.

On not complying with FOIA requests:

From: Phil Jones
To: “Michael E. Mann”
Subject: IPCC & FOI
Date: Thu May 29 11:04:11 2008

Mike,

Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise. He’s not in at the moment – minor family crisis.
Can you also email Gene and get him to do the same? I don’t
have his new email address.
We will be getting Caspar to do likewise.
I see that CA claim they discovered the 1945 problem in the Nature paper!!
Cheers
Phil

Prof. Phil Jones
Climatic Research Unit Telephone +44 (0) xxxx xxxxxx
School of Environmental Sciences Fax +44 (0) xxxx xxxxxx
University of East Anglia

Note: “Michael E. Mann” refers to Dr. Michael Mann, who popularized the Hockey Stick temperature graph.

Other information is also very interesting. There is a spreadsheet showing how much grant money Dr. Jones has received since 1990: £13.7M. An email shows that there was big money flowing around:

At 11:31 13/10/2008, Tim Osb*rn wrote:
>Hi CRU Board,
>
>I just had an interesting chat with Jack Newnham
>from the International Development Team at Price
>Waterhouse Cooper. They get lots of DfID
>(Douglas: DfID is the UK Government Department
>for International Development) funding.
>
>They’ve heard that DfID are likely to call for
>expressions of interest for a new centre
>focussing on international climate
>change. Their idea is to fund a centre that
>would be the first point of call for advice and
>for commissioning research related to climate
>change and development or to climate change in countries where DfID operate.
>
>He was talking about £15 million per year for 5
>years! Not sure how much would be from DfID and
>how much raised from other donors (and hence
>uncertain), nor how much would be given up-front
>versus how much spent later on specific research
>projects organised via this centre.
>
>Nevertheless, sounds big enough to be worth getting involved in.

That's £75M in one shot right there. Adding in Dr. Jone's funding, and you have about £100M identified in just these two files.

Lastly, source code and what appears to be raw data sets appear to be in the released data.

What is the likely short-term impact of the disclosure?

The most significant impact is probably in the public debate. While the public has a very low appetite for esoteric scientific arguments, personal conflict is a very different matter. Given that more than a little of this looks dodgy (just why are they trying to get around FOIA requests), and since this is the first whiff of just how much money is floating around the Climate Research community, and you have the making of a story that the general public might be able to sink its teeth into.

Given that the general public doesn't believe the Anthropogenic Global Warming hype anyway, this seems like the most significant short term impact - the narrative becomes not "dodgy science", but rather "dodgy scientists".

We'll have to wait and see how the press plays this. The interest is ramping up all over. Follow the money, indeed.

What is the likely long-term impact of the disclosure?

Just what's in the source code and data? It will take a week or a month for skeptical scientists to start really examining these, but we'll soon start seeing every mistake, every computer programming error, every hard to justify model assumption publicized. We'll also see any data errors or inconsistencies, as well as divergence between the models as run my the skeptics and model results as published in the peer-reviewed literature.

In other words, scientists will do what scientists normally do, and try to recreate published results.

Again, this is not based on my own examination of the data. I'll post on the data tomorrow.

UPDATE 23 November 2009 10:22: JayG linked, from a very interesting post about how the MSM is playing this. Cliff's Notes version (cue Mark Twain): If you don't read the newspaper you're uninformed. If you read the newspaper, you're misinformed.

UPDATE 23 November 2009 23:16: Paul The Pirate linked, with the logical question:

To me, this has little to do with science, and everything to do with the horrific results of politicization of virtually anything. Science, medicine, ethics, whatever. The same people who gave us eugenics, social Darwinism and the other bastard children of science are at work again. Last time, it was 'only' minorities and the handicapped who suffered when smart people used fake science to fool the not-so-smart. Who's going to be on the shit end of the stick here?

Sadly, we know the answer.