Why the Polls Were Wrong

It seemed like a sure thing. National polls put Republican John McCain several percentage points ahead of Mitt Romney in Tuesday's New Hampshire primary-and they were right. McCain's win, with 37 percent of the vote, resurrected his fledgling campaign and threw the race for the Republican nomination wide open. But on the Democratic side there was a disconnect. Polls showed that his Iowa victory had propelled Barack Obama to a significant lead over Hillary Clinton in the run-up to the New Hampshire vote. But when the dust cleared, Clinton had eked out a stunning three-point upset. So why did the surveys get it so right in one contest and so wrong in the other? NEWSWEEK's Jessica Ramirez asked Frank Newport, editor in chief of the Gallup Poll. Excerpts:

NEWSWEEK: Before Tuesday, polls gave Obama an eight-point lead, on average, over Clinton. Your own polls, conducted through Sunday, put that lead in the double digits. What was your reaction when you saw the eventual result?
Frank Newport:
Generally speaking, our polls and other well-done polls have been accurate predictors of what people do on Election Day, both in primaries and nationally. Given our poll, which we had confidence in showing Obama with a sizable lead, and every other poll that showed Obama with a lead, my expectation as a professional was that in New Hampshire Obama had the highest probability of winning. So I was surprised by the results, no question about that.

So how did it happen?
We're looking into that carefully. We don't think that there was a flaw with the polling per se. By that I mean a sampling or methodological problem. The poll was well done. That's underscored by the fact that on the Republican side the estimates were remarkably close to what actually happened. We had McCain up by four, and he won by five. That leads us to believe there was something going on with Democratic voters in New Hampshire, and that it wasn't actually a problem with the polls themselves.

There are a few theories on why Hillary surged. The main one seems to be her emotional moment. What do you think?
It is a reasonable hypothesis that it helped her. I think it's quite reasonable that Democratic voters were continuing to evaluate Monday and even into Tuesday. We're actually going to be doing some reinterviewing in New Hampshire; we're going to call back people that we interviewed and get more details on the process of their decision making. We're continuing to follow what happened and what perhaps changed voters' minds.

What do you hope to learn from going back and interviewing again?
We're doing it for two reasons. One is to find out more about what happened in New Hampshire. We also want to know what's going on with these people who have gotten to know these candidates intimately, so to speak. We want to know a little more in depth what they think about them, which will help us understand why voters-even elsewhere-may decide to support either Hillary or Barack.

Let's say Hillary's "moment" is why the numbers were off. That event happened on Monday. Most polls were complete by Monday. Is there anything that can be done to account for last-minute shifts in the future?
We sometimes, in the national polling in November, do poll all the way through Monday night. One of the reasons that we don't, typically, in New Hampshire-and other pollsters may not poll through Monday nights-is because the data has less value to consumers. One of the reasons we have polls with our partner USA Today and for our own readers at Gallup.com is that people are interested and want to know what's ahead. When we can provide them that data late Sunday night and Monday it gives us a full day to kind of explain and go over the data. If we poll all the way through Monday night, it might be satisfying to us, but by Tuesday there's less interest. Voters are turning to the actual voting going on by then. That's one of the reasons we have not. On the Republican side it didn't make a difference. We stopped on Sunday and what Republicans told us they were going to do on Sunday is absolutely what they did on Election Day. More often than not there's not a lot of huge last-minute vote changing, but it can happen.

Since it happened in this case, is that going to influence how close to a primary your organization will poll?
It certainly may. Our scientific procedures here at Gallup are an evolving process. We're constantly looking for ways to improve. In this situation we are analyzing everything we did. We will consider very carefully, when we poll in a future primary state, continuing to poll through Monday night. We'll take what happened in New Hampshire into account in making that decision.

Has there ever been this defiance of poll numbers during such a critical primary before?
I don't have all the data in front of me, but it's rare that all of the polls show that one candidate is leading going into a primary-in many of them, including our own, by such large margins-and then the other candidate wins. It's unusual. That's why I was surprised.

Are there any post-New Hampshire signs that we might see this sort of defiance again?
Well, we don't know. We'll have to see. We'll monitor carefully and be very sensitive. I'm sure the media are going to be very sensitive too. I'm sure the gatekeepers of the media will downplay reaching any major conclusions based on pre-election polls. Usually polling is quite accurate. It's an important component of our election process. We'll continue to poll and, hopefully, for the most part, we'll be able to document what people are telling us is what they do on Election Day. If not, then we've learned something. We've learned that we've moved to a faster-changing environment. That this year, in '08, voters will turn; that they are not set in their ways.

According to Gallup data, it's rare for Iowa and New Hampshire to vote for different Democratic candidates. What could this surprise split mean for the overall race?
Well, it's arguably a different situation than, say, in '04, when one candidate won both states and was kind of on his way. They split the first two, and even our poll after Iowa nationally showed Clinton and Obama were tied. It all suggests that the race is far from over and there's much more excitement to come. I think that's what this tells us.

On the GOP side, McCain won, as polls predicted. Why was that win so in line with the numbers?
That's the norm. That's typical. We didn't poll in Iowa, but the people that polled in Iowa found roughly the same thing the week before. They polled, got estimates of voting, and that's what played out in the caucuses. The fact that on the McCain side, the Republican side, the actual voting behavior was close to our weekend poll was not surprising.

His victory has opened up the GOP race even more. Could that lead to any future New Hampshire-like results?
Not necessarily. We had no problems in Iowa or New Hampshire on the Republican side. So I wouldn't think the fact that it's close among these candidates will produce any unusual challenge.

Should we expect the "comeback kids" to bump the current presidential front runners in the polls?
Everything else being equal, I would think in national polling Hillary Clinton will get a bounce. Since she was tied with Barack Obama prior to New Hampshire, there's every expectation that she will lead. On the Republican side, McCain was a few points behind Huckabee and essentially tied with Giuliani. I would expect McCain to gain, but to what degree, we'll just have to wait and see.

Join the Discussion