Poll discovers Journalists think that Math is hard

OK, I just made that up, but it's pretty clear that Journalists simply don't understand statistics. Well, Stats is hard (I thought it was harder than Calculus, but that's me). However, it's not asking too much for them to have someone who does know stats to check things out for them, is it?

Because otherwise, they won't realize that the current polls are mathematically impossible:

The variance for current polls listed at Real Clear Politics for this morning ranges from the Pew poll which advertises a 14-point lead for Obama, to the Battleground poll which says the lead is only 2 points. The variance is too great (and there are polls relatively close to both ends, demonstrating proof of statistical invalidity for the published confidence level) for even the casual observer to accept as a reasonable. There are four polls which show a 10 point lead or greater for Obama, and another five which show a 6 point lead or less. It is mathematically impossible for so many polls to be valid, yet disagree to such a degree with valid methodology. [emphasis in the original]

Lots of discussion of statistical confidence intervals in the post, but it's mostly painless. The key parts, though, are about how the pollsters (Gallup, in this case) are just making up the methodology:

Gallup has admitted that this model has no precedent, and uses over-samples of urban and youth voters, in the presumption that they will sharply increase participation this year.

Other than the "this model has no precedent" bit, it's rock-solid.

Other pollsters seem to be similarly "rock-solid":

For example, in the Oct 20 poll Pew undersamples seniors and oversamples the 50-64 age group, oversamples high school only education by a large amount, and fails to note regional breakdowns or the urban/suburban/rural split. These are critical points which Pew fails to address, and which hshould [sic] make the reader wary.

JD Drummond has a must read about lousy polling methodology, and concludes that while the press is happy to be fooled, the candidates aren't acting like they are:

So, could I be wrong? I have to be honest and admit that I could. But in that case, we'd have to ask why the polls do not generally agree with each other, why Gallup is trying to spin three different models at the same time to get a grasp of the picture, why McCain and Obama are both so interested in Pennsylvania, yet neither is working very hard in Ohio right now. We'd have to explain why McCain-Palin rallies are now attracting thousands more people than Obama-Biden rallies, why Letterman suddenly found it cool to have McCain on his show and SNL decided they wanted Palin on theirs. We'd have to explain why there are not a lot of Obama signs visible, but we hear about his army of lawyers getting ready. We'd have to explain why McCain and Palin appear to be so relaxed while Obama and Biden look like they're worried.

So what does this all mean?

1. I stand by my earlier post. I don't believe that the polls are valid. I'll even add to that, saying that I don't believe that the actual election results this year will be within the margin of error (IOW, they're all a crock).
2. The pollsters are confused. Gallup, for example, uses three different and mutually exclusive weightings, none of which are historically valid, nd none of which agree with each other. See the Stolen Thunder link for details, but this reinforces point #1.
3. The press doesn't have a clue that the polls are worthless. Whether this is due to mathematics ignorance or willful ignorance is left to the reader's imagination.
4. The press may be trying to walk back from the ledge. Whether they can resist the temptation to double down on their hopey-changey candidate remains to be seen.
5. The candidates know better. You want to know what's really happening? Watch them. No way is Obama up by 14 in Pennsylvania - why would he campaign there if he were?
   

It seems that a lot of this campaign has been about "fool the rubes." The polls, and the press' flogging of the polls fits this pattern. Ignorance or willful ignorance?

UPDATE 22 October 2008 19:27: OK, I'm slow on my blog reading today, because Kim du Toit sums the situation up with the Quote of the Day:

The Cliff Notes explanation is that if I’d ever designed a commercial qualitative research study with so obvious a set of prejudices and misconceptions, I’d have been fired.

UPDATE 22 October 2008 21:15: It just keeps on coming. Zombietime has a post about the "Clever Hans Effect" as a source of polling bias.

This phenomenon is now called "The Clever Hans Effect": the attitude of questioners can affect respondents' answers even if the questioner is trying to remain neutral. If a horse can notice subtle verbal or visual cues, then a person -- who is much more attuned to human culture and emotions -- can probably notice them even more so (possibly also without even realizing it).

I'd really like to see an analysis on what this does to the confidence interval and margin of error. Interestingly, the CE does not relate to accuracy, but rather to repeatability. In other words, you can have a perfectly repeatable poll stuffed full of, well, what comes out of Clever Hans.

Polls. You keep using that word. I do not think it means what you think it means.