Why Were the Polls So Wrong?

It’s not just shy Trump voters.

An election worker takes ballots from a sorting machine in Renton, Washington on Election Day.
An election worker takes ballots from a sorting machine in Renton, Washington, on Tuesday. Jason Redmond/AFP via Getty Images

On Thursday’s episode of the Political Gabfest, the hosts discussed why polling failed so miserably in 2020. This partial transcript has been edited and condensed for clarity.

David Plotz: Maine’s Susan Collins did not lead in a single poll in her Senate reelection race, and she won it pretty easily. Joe Biden was up by double digits in much Wisconsin polling, and it looks like he’s going to win that state narrowly. Michigan and Florida polls also favored Biden out of proportion to the final results. John, after the 2016 election, there was a tremendous reckoning about the polls having been wrong, and pollsters went out to correct it and to sample more non-college-educated white voters in particular, to make sure they were balancing for education better. Yet it looks like the error is even worse in 2020. Any thoughts on how we should start to think about this?

John Dickerson: Before we knew about polling misses that helped Democrats, there were polling misses that helped Republicans. In 2012, there was a lot of concern about Republican polling. So the thing to question is whether there’s always a polling miss and there is no particular bias to the miss, or whether there’s an ideological bias to the miss. That’s one set of questions. The second is whether there’s a particular kind of miss when it comes to Donald Trump. And then there are always misses, because stuff changes. For example, my view that was informed by lots of scholarship on the power and importance of debates is evolving for one idiosyncratic reason and one that is structural: Donald Trump operates outside of the normal behavioral patterns, and therefore his debate performance was just so out of control that it had an effect, whereas normally candidates don’t vary that far from the median, and therefore, debates don’t play a role. I think Donald Trump’s debate performance did play a role. Secondly, because of early voting, it played a role because people could vote right after it. So they could take an opinion about Donald Trump, say, “I’m done with him,” lock in the vote, and that’s it.

My point is that views we have about voter behaviors and patterns are always changing based on the circumstances. We need to be alive to that.

But here you had polling that was messed up all up and down the line—and they got it wrong in different places and in different ways. Was there really a shy Trump voter? Did people misunderstand the inroads that Trump had made in the Latino community in Miami-Dade County? I think we’re going to have to figure all that out, but the miss doesn’t appear to be about people not saying they want to support Trump, because whatever the polling error was, it was the same for Senate candidates and House candidates, Republicans generally. It’s that they missed a ton of people who were going to vote Republican.

Emily Bazelon: If you have polls in close states that miss by 4 or 5 points, they’re effectively useless. And if you have polling averages and then probability models based on those averages, then it’s really hard to see how this is introducing some kind of rigor, even though it has had a huge impact on political journalism and on how we all talk about these races.

One theory of this, which I think comes from David Shor, the data analyst, is that people are much less likely to pick up the phone and answer the poll if they have low social trust. So you wind up with an unrepresentative sample of people, because it turns out Democrats or people who support Democrats are more likely to have social trust. If you combine that with the incredibly low response rate for simply answering the phone, maybe we just can’t do this anymore in an effective way. We need to have more ways of understanding the electorate, because this one is just insufficient.

Plotz: It has an actual effect. Think about, say, the Jaime Harrison–Lindsey Graham Senate race in South Carolina. There are reasons people put money into that race in general: A lot of Democrats really wanted Lindsey Graham out, and the idea that it was within the realm of possibility inspired people to overinvest in that. But if the polls had been accurate, it would have been crazy. Every dollar people put into Jaime Harrison’s campaign would have been better spent in Georgia. It actually misleads the professionals and causes a wrong allocation of resources and probably ended up costing the Democrats a bunch of seats.

Bazelon: You have to be a little careful, because otherwise you have a situation in which the only people who poll well are the people who look like traditional politicians, i.e., white men. So then you never get the kind of investment in Black candidates or in women that you would otherwise want.

Plotz: Well, no, but it would have said South Carolina is an incredibly hard state for a Democrat to win. It’s going to take something incredibly big to change that. Georgia, less so, because the demographics of Georgia are different. So let’s put money in states that are more accessible, and let’s not get taken by this mirage that a poll can give us.

Bazelon: When you bring up Georgia, I have to bring in all of the painstaking work of registering people of color to vote across that state that Stacey Abrams did before she ran for governor and that she has continued to do. What you’re talking about, David, is the difference between investing in on-the-ground work that is very incremental. It’s hard to see the gains from it right away. But if you don’t do it, then you never change the whole complexion and set of possibilities of the electorate.

Dickerson: Also, people were going to do crazy stuff regardless of the polls. I mean, Mitch McConnell was never really in danger in Kentucky, and a boatload of money went into that state. So it’s not just the polls that are motivating a lot of the money that goes into these races. I don’t know that you can put all this on the pollsters.

Plotz: Polling is a clever way to keep people’s attention. I’m not sure what you would put in its place, because it is just as misleading to do anecdotal reporting, where you go and talk to a Trump voter or a Democratic voter, and use that one person to signal something socially significant. That isn’t any more accurate. It’s less accurate.

To hear the entire episode, in which the hosts also discussed Trump’s election lawsuits (with guest Nate Persily) and Democrats’ down-ballot disappointments, subscribe to the Political Gabfest on Apple Podcasts or listen below.