Just how lousy are the climate models?

I posted this ten years ago, and it is still as relevant today for the ZOMGTHERMAGEDDON!!!!11!!!eleventy!! hype.  It also is important to chew on when you look at the models for Kung Flu deaths.  A healthy skepticism is always in order when you hear reports of model projections.

Remember, all models are wrong; some are less wrong.  Your mileage may vary, void where prohibited, do not remove tag under penalty of law.

Originally posted April 1, 2010.

Climate Model software

There's a very interesting comment that I missed last spring, over at an Armed and Dangerous post. Ken Burnside writes on how they work, in some detail.

1. Ken Burnside Says:
   April 19th, 2009 at 11:05 amI have old-but-direct experience with the models, circa 1999, about the time of the Mann graph.
   
   I was a science reporter in Madison, WI. And got asked to interview Dr. Reid Bryson at the UW Madison. Bryson is, for all intents and purposes, the figurative founder of atmospheric and climate sciences.
   
   I wouldn’t know where to look for the bugger factors in the models. 
   
   However, he and I spent about 3 weeks running three of their models through their paces.
   
   We started using the data sets they gave, and got the results they indicated.
   
   We then looked at the data sets given and compared them to historical temperature proxies, and found some discrepencies, and noted that it treated the H20 storage capacity of the middle atmosphere as being infinite (major problem).
   
   We then ran historical data from 1900 into all three models, and let them run (each modeling run took about 4 days, so we ran them in parallel.
   
   One had the oceans boiling off in the 1950s, because the temperatures in the 1930s triggered a runaway greenhouse effect.
   
   The other two were less spectacular – they had temperature rises of about 5-6 C and 6-9 by the end of the century.
   
   We then tried to isolate the forcing factors and see what they were; we ran the most extreme model with a solar input constant that was HALFED (EG, we did the equivalent of moving the Earth to Mars’ orbit). We postponed the boil-off effect to the late ’90s by doing that.)
   
   So, we take the data sets they give, we run corroborations, we can’t replicate the historical record.
   
   So, who is it that doesn’t understand the scientific method? 
   
   Atmospheric observational scientists don’t report anything CLOSE to what the predictions make.
   
   The Dean of the American Society of Statistical Sciences (Wegman) says, in essence, that if the statistical methods used by most of the climate sciences were used that way on a Freshman stats class, he’d flunk them all – and says, before Congress, that they cooked the books for a political agenda.
   
   I write models for a living – I design games. This doesn’t mean I’m up to all the tricks of cooking models that are out there, but I do know how to do bounds checking on them, how to run a chi-square, and look for hot spots. I have discovered that I’ve got more day to day use of my stats and calculus classes than most of the professional working scientists I know, or have as customers.
   
   If I’d gotten a historical wargame submission this bad, I’d've sent it back with a reading list on the topic and told the author to try again.

That's six months before ClimateGate.

The whole post and the entire comment thread makes for very interesting reading. Very interesting. If this is your bag, baby, then you should go look. Plan on a good 15 minutes or more - there are 266 comments.