Arrogance

Ever wonder how someone makes a computer model of a really complicated natural system? Say, the World's Climate? You make a lot of simplifying assumptions. Micro effects are chaotic and can't be mapped to macro effect? Replace it with a seat of the pants estimate. You're an atmospheric physicist but you need to model oceanic thermal transfer? Well, air is a fluid, sort of.

But if some snotty-nosed blogger asks to see your source code or data, you get offended. After all, you were always the Smartest Kid In Class. You're published. In peer-reviewed Journals!

The problem for the in-group is when the out-group is as smart as they are:

I was trained as a physicist and was granted a PhD for my postgraduate work in upper atmosphere physics. In the early 1980s I joined the CSIRO’s Division of Oceanography and worked in surface gravity waves (ocean waves) for a time. Much of the theoretical side of oceanography entails fluid dynamics which, because of its heavy mathematical load, is regarded as a sub-discipline of applied mathematics rather than of physics. Because of this, in my view, many practitioners of oceanography and climatology have a cavalier disregard for experimental testing and an unjustified faith in the validity of large-scale computer models.

Later in my career I was involved in running and refining numerical fluid dynamical models, so I gained some insight into how this modelling is done and how rigorously such models need to be tested. Naval architects and aerodynamical engineers do such testing in wave tanks and wind tunnels.

Meteorologists regularly test model “skill”. Climatologists don’t seem to have a concept of testing, and prefer to use the term “verification” instead—that is, they do not seek to invalidate their models; they only seek supporting evidence.

Sheesh. What would an Atmospheric Physicist who worked on Oceanography know about how the atmosphere and oceans effect the climate? Must be some sort of denier, or something. Reid continues:

Back in the early 1990s when I was still working for the CSIRO and the early versions of the AGW theory started to gain currency, I was rather bemused by the passions which were aroused in my colleagues and the gullibility with which predictions of future climate disaster were accepted. Surely the jury is still out, I thought. I remained agnostic about the theory. More recently, after reading the literature and looking in detail at the output of one well-known climate model (HadCM3) I have changed my stand. I now believe it is nonsense for the following reasons.

[snip]

Second there are the climate models themselves. In discussions with colleagues, arguments always seem to come down to “But the models show …” Those who use this argument seldom have modelling experience themselves and share the lay public’s naive faith in the value of large computer models.

I have been a fluid dynamical modeller and I know how flaky numerical models can be for even a relatively small chunk of fluid like the Derwent Estuary. The models are highly unstable and need to be carefully cosseted in order to perform at all realistically. One reason for their inherent instability is that the mesh size of the model grid (typically hundreds of metres to hundreds of kilometres) is always much larger than the scale at which friction and molecular diffusion operate (millimetres or less). These are the forces which act to damp down oscillations by converting free energy to heat. In order to get around this difficulty, in order to keep a model stable, it is common practice to set certain parameters such as eddy viscosity unrealistically high to compensate for the absence of molecular friction. This is reasonable if we are using the model to gain insight into underlying processes, but it means that fluid dynamic models are not much good at predicting the future. There is no exact correspondence between model and reality, and the two soon part company.

It's too hard to model the fluid dynamics of one river mouth, but the climate of the entire world? Don't worry, they have it covered.

I must admit here that I'm certainly no climatologist. However, my entire industry (Internet Security) exists because - I'm afraid that there's no good way to say this - all programmers are lousy. If it weren't for bugs (and most of the time, it's bugs that we've seen again and again and again), you wouldn't need our products. There are two types of bugs:

1. Programming mistakes. In security, these aren't bad enough to make the program crash (or someone would find in during Quality Assurance testing, duh). These are the easy bugs. Something's broken, and needs to get patched.

2. Architectural flaws. It's not a bug, it was designed that way. These are bad security juju, because the security "bug" might not be fixable at all, or it requires heroic efforts to fix. Programmers hate these bugs: not only are they a pain in the tail end to deal with, but the other programmers laugh at them because their program is broken as designed.

So, are there any bugs in the models? Speaking professionally, we should expect roughly one security bug per 1000 lines of source code. How many lines of source code are there in something that models the climate of the earth? We don't know, because the scientists won't publish their source code. But don't worry, I'm sure it's all wicked smart and accurate and everything. And well tested. Oh, wait:

A scientific theory is not tested merely by looking for confirmations but by conscientiously trying to “break” the theory, by trying to disprove it. The AGW theory is encapsulated in the IPCC assessment reports. The models discussed in these reports have not been tested in this way. These reports include sections on “Verification and Validation” but none on testing. “Verification” means that only data which support the theory are examined and data which do not support it are ignored. Indeed the authors of this section in the IPCC Third Assessment Report specifically dismiss the need for rigorous testing when they state: “our evaluation process is not as clear cut as a simple search for ‘falsification’” (Section 8.2.2 on page 474). Effectively what they are saying is: proper scientific testing is too hard and we are not going to bother doing it.

Testing is hard, and often thankless. And really, really important. A company that doesn't test its products goes out of business, because customers won't put up with lousy output.

Go read the whole thing. This is one of the most interesting articles on the science behind AGW that I've seen in a while, and I look for this sort of thing. The quality control of much - if not most - of the AGW research looks terribly shoddy.