Saturday, November 1, 2008

Trust Exit Polls at Your Peril

Jim Geraghty posted this yesterday:

McCain Voters Less Likely To Respond To Exit Pollsters

Obviously, it's tough to measure the attitudes of those who refuse to talk to pollsters, but this question in Fox News' latest survey confirms a long-held suspicion [...]

Now, I'll tell you now - polls close first in Kentucky, Indiana, Georgia and Virginia at 7 p.m. eastern Tuesday . I find it extremely likely that Indiana and Virginia will be called for Obama immediately after the polls close by at least one network, based on these exit polls that have fewer pro-McCain respondents.

The ingredients are there for a rerun of 2004, when the exit polls indicated a Kerry victory and deviated signficantly from the actual results.

Most interesting. Let's put this in the context of a piece Rick Ballard wrote at American Thinker:

It is my belief that the story from Marion County will become the equivalent of the early call of Florida for Gore in the 2000 election. I expect to see early stories of "massive and unexpected turnout in Indianapolis" all day long with a very early call (like 6:01) of Marion County for Obama, followed by "if McCain is doing this poorly in Indiana, it's all over". The whole purpose of such a story will be to suppress Republican turnout in states where the polls are still open, particularly in Colorado, New Mexico and Nevada.

There is no question that this election is going to be tight. There is also no question that it will be the Republican Get Out The Vote effort which will make or break John McCain's chance of winning. There is no question that Obama, Axelrod and the media will be working hand in glove all day Tuesday to generate as much positive noise about the inevitability of an Obama win as possible. There is a solution to the problem: "Now get out there and vote, dammit"

Yes, Florida in 2000, and the election of 2004, let's just say there's a pattern. Speaking of 2004, when the first exit polls came out, we were subjected to reports that John Kerry was up big in key states. Liberals and Democats were ecstatic, Conservatives and Repulicans were despondent. Both swallowed what the poll data was selling. The Kerry campaign began its premature victory dance:

Kerry spokesman Mike McCurry said Friday that on election day, as Kerry was conducting four hours of local television interviews aimed at increasing last-minute turnout, McCurry and top aides John Sasso and Bob Schrum were receiving wildly positive reports from their state directors and encouraging exit polls on their Blackberrys.

McCurry said they didn't dare tell Kerry the good news, for fear he might stop the interviews, but Schrum could not contain his optimism the moment the candidate had finished.

"Let me be the first to call you Mr. President,'' Schrum said to Kerry.

(and proving that he does not learn from history, Shrum wanted to be the first to call Obama Mr. President, too)

Bob Shrum was not only the first person to call John Kerry "Mr. President", he was the last -- apart from those deluded or those making jokes at his expense.

But listen. We're in for it again on Tuesday. Mark my words. Gird your loins. Trying to sift through innacurate exit'll be like cleaning the Augean stables, man.

Rest assured, we will hear stories of Obama winning, and winning big, in exit polls. And as Mr. Ballard mentions, it will be used in a manner to help write the narrative that an Obama win is upon us, inevitable, imminent, that there is no need to vote, McCain supporters out west might as well stay home.

But raw exit poll data is seriously flawed for the reason that Geraghty points out. Let's take a look at the numbers. I went to the Fox survey so I could get the break downs for Very likely, Somewhat likely, Not very likely and Not likely at all categories for both candidates in terms of responding to exit pollsters.

Very likely = 46%
Somewhat likely = 31%
Not very likely = 9%
Not likely at all = 11%

Very likely = 35%
Somewhat likely = 29%
Not very likely = 16%
Not likely at all = 16%

Next, let's assign an assumed response rate to each category to represent the percentage of those who would actually participate in an exit poll...

Very likely = 87.5%
Somewhat likely = 62.5%
Not very likely = 37.5%
Not likely at all = 12.5%

NOW...applying the value for the percentage of Obama and McCain supporters who say they are in each category to the response rate of each category, you can estimate the number of voters who would actually participate in an exit poll.

Exit Polling: Who Responds

That is, if you asked 100 Obama voters to participate in an exit poll, you would get the following totals…

Very likely = 40.3
(out of 100 Obama voters polled, 46 of them would be in this category and 87.5% of them would agree to participate)
Somewhat likely = 19.4
Not very likely = 3.4
Not likely at all = 1.4
TOTAL = 64.4

Out of 100 actual Obama voters, 64.4 would agree to participate in the exit poll.

And if you asked 100 McCain voters…

Very likely = 30.6
Somewhat likely = 18.1
Not very likely = 6.0
Not likely at all = 2.0
TOTAL = 56.8

Out of 100 actual McCain voters, 56.8 would agree to participate in the exit poll.

Out of 200 actual voters asked, 121.2 would agree to participate in the exit poll.

NOW…let's say you were to ask 1,000,000 people to participate in an exit poll in an election where the vote was split exactly 50/50…

Exit Polling: 50-50 Scenario

Obama -- 53.1%

McCain -- 46.9 %

Obama would have a lead of 6.2% in the exit poll when the vote was actually 50/50.

Or if you reverse the process…and had the same situation -- but the election itself was McCain 53.1% and Obama 46.9%...the exit polls would show 50/50!!!

Exit Polling: 53-47 Scenario

Watch those exit poll numbers as they come out Tuesday evening. In the hands of competent and objective analysts, who account for the likelihood that Obama voters will participate more than McCain supporters, there can be some great information about who is winning.

In the hands of an Obamaphilic media eager to write the narrative that Obama is winning in order to suppress votes?

Be prepared to endure a great amount of disinformation.

And no matter anything else, let's return to Mr. Ballard's admonition:

"Now get out there and vote, dammit"


  1. All of this loin girding is very stimulating...

  2. Don't give it to him. Make him steal it.