On this week’s episode of my podcast, I Have to Ask, I spoke to Nate Silver, the election forecaster and editor in chief of FiveThirtyEight.com. Below is an edited excerpt from the show. In it, we discuss how the fight over Brett Kavanaugh’s nomination might impact November’s election, the ups and downs of Silver’s tenure at the New York Times, and why the 2020 Democratic presidential primary might look a little bit like the GOP’s 2016 circus.
You can find links to every episode here; the entire audio interview is below. Please subscribe to I Have to Ask wherever you get your podcasts.
Listen to I Have to Ask via Apple Podcasts, Overcast, Spotify, Stitcher, or Google Play.
Isaac Chotiner: What have you made of the polls about Kavanaugh, both about what voters make of him and also about what effect, if any, this whole debacle has had on the views of the two parties heading into the elections?
Nate Silver: I think from a top-line perspective, No. 1, Kavanaugh himself is fairly unpopular. Although a lot of things Republicans do are fairly unpopular. So is he more unpopular than Trump or the health care bill or whatever else? Maybe not. But he’s underwater by 6 or 7 or 8 or 9 points depending on what poll you look at.
However, in the top-line numbers, I think you can argue there are some good news for Republicans, at least in the Senate. So the Senate, it’s much more granular. It comes down to individual races. And we’ve seen, in the past 48 hours, two polls showing Heidi Heitkamp down by double digits in that Senate race in North Dakota.
The House is a little bit more ambiguous. It’s a weird case where I think the conventional wisdom is half right. The conventional wisdom being there’s a big surge now in GOP enthusiasm that has been a game-changer potentially in the outlook. I think that hypothesis is more or less exaggerated when it comes to the House but has enough validity in the Senate.
Just to take a step back: After the 2016 election, there was a ton of criticism of people who do what you do. How do you think the polls were or were not incorrect about 2016? And how do you look at your 2016 model now?
So I think our model did awesome in 2016. I think it probably did better in 2016 than 2012, the reason being because it detected a lot of important but fairly subtle qualities that led the model to believe that Trump had a reasonably high chance—I think it was 29 percent; you can round off to 30 percent—on Election Day. One of those was the fact that you had a lot of correlation from state to state, so the fact that Trump overperformed his polls in Wisconsin and Minnesota and Michigan and Pennsylvania. He didn’t win Minnesota but still beat his polls there. That’s not a coincidence. It’s all about the white working-class vote and how Clinton’s lead was not very robust, because all you needed was for her to underperform among one group—white working-class voters—and that was enough to topple her Electoral College position.
Basically, our goal is to make forecasts, to handicap the race, and to lay odds. There is a lot of work in setting the odds correctly. And a forecast that gives Trump a 3 in 10 or a 1 in 3 or a 1 in 4 chance is a lot different than one that gives him, say, a 1 in 100 chance. And so if you were actually making bets based on our forecasts in Vegas or in a prediction market, then you would say these guys have Trump much higher than prediction markets do, than other forecasts do, than the conventional wisdom does. And so you would have bet a lot on Trump. And I think you would have bet a lot on Trump because we think a lot about uncertainty and how we design the model, about the statistical characteristics of polling error, and so forth.
And so for us, this is a case where we think the model added a lot of value. Now is the average reader out there going to think that? I don’t know, but in some sense, I don’t really care. We’re trying to do the best forecast that we can. It’s something where it’s fairly technical material, which is broadcast to a very wide audience, and that means people are encountering mathematically complex concepts that they might not encounter as much day to day. But all we can do is build the best forecast we can. I think our thoughtfulness about how we design models was reflected in the fact that we gave Trump a much higher chance than other people did. And so, I don’t know.
How is your model different this year, if at all?
Philosophically, there’s no difference, because, again, we think that first of all—look, if you have a model where you have a 1 in 100 chance of something occurring, yeah, maybe you got unlucky, but then you probably ought to rethink things. The flip side of this, you shouldn’t be doing a lot of rethinking when a 1 in 3 or a 3 in 10 chance comes through. That’s almost irresponsible and not true to the whole point of spending all this time thinking about, Boy, how do we model and measure uncertainty? And that’s a lot harder than estimating to say, Oh, Clinton is ahead 2 points in the state. So philosophically, there’s not as much of a rethink.
How worried are you, especially in some swing states, that the polling error had to do with underrepresentation in the polls of voters at lower education levels? And how worried are you that that’s going to repeat itself, because American politics is increasingly polarized along lines of education—especially with white voters of higher and lower education levels—increasingly going in different directions?
I’m worried about that. And I’m equally worried about pollsters overcompensating and/or potentially missing sources of error that would lead them to underestimate how Democrats would do. One of the more robust findings in our years of doing this is that polling error is not directionally very predictable. Meaning, will the overall forecast be off? Yeah, it very easily could be. It will be off to some degree. It’s just a matter of how far it will be off. But it’s equally likely to be off in either direction.
For example, in California, Hillary Clinton beat her polls by 7 points, which is more than she underperformed her polls in Wisconsin by, which was I think 5 points. The importance of that is that there are seven or eight or nine competitive House races in California. Democrats’ polling has been tepid there. If that’s a polling error, that could be an example of where Democrats actually pick up a few more seats in the House than projected by the polls.
For a long time, you had a critique of political journalists and the way they wrote about politics and the way they wrote about polls. I’m curious if you can just lay out what that critique was for people. How relevant do you think it is today? Do you think those errors have been corrected?
The basic critique is that there’s a lot of groupthink among journalists that involves a lot of cherry-picking and confirmation bias and is often quite divorced from the underlying reality of a race. The second area of critique might be that, hey, there’s a lot of innumeracy, so with respect to explaining probabilities, margins of error, concepts of the Electoral College, and so forth, that tends to color things as well.
I’m not sure that that critique has lessened at all. In fact I think, again, one of the things about 2016 that annoys me when people take shots at pollsters or forecasters is if you had surveyed the average New York Times beat reporter or Washington Post—we’ll be inclusive here, Politico, etc.—and asked them, “What do you think the chances are that Trump will win? Are they north or south of 30 percent?” I am positive that the large majority of them would say way south of 30 percent. And so, in all the cases where the polls have been “wrong,” or at least a few points off, general convention of wisdom has been equally, if not more, off. And so I’m not sure that’s corrected itself.
I do think you have more numeracy overall. I think journalists now are smarter about not looking at just one poll and waiting for two or three polls to confirm a trend. Although with this Kavanaugh thing, this feels a little bit more old-school, where people are very quick to jump on this Republican-enthusiasm-surge narrative, and so are maybe cherry-picking a little bit.
But look, I think fundamentally there is an opposition between the way FiveThirtyEight looks at races, where oftentimes it’s kind of a slog, right? A little bit of data comes in at a time. The changes are not all that large from day to day. I think it’s fundamentally pretty different from the up-and-down kind of horse-race method of reporting.
From following you on social media, it seems to me that you have some specific critiques of the way the Times—where you used to work—covers politics. Is that fair?
I have critiques of how the Times covers elections.
Which is what?
Which is I think they are way too reliant on … first of all, in 2016 they were very, very subscribed to the idea that Clinton couldn’t lose, and she couldn’t lose because the Electoral College favored her. Why were they subscribed to that? No. 1, I think the Times is not skeptical enough about insider narratives, and that’s probably what they heard from their sources and they didn’t critique them as much.
No. 2, there is this firewall between the Upshot and the rest of the newsroom there. I think that’s been lessened a little bit, but it’s a very kind of narrative-driven place. And they want to take what they see out there and tell stories with it—stories that a lot of the time are oversimplified in their campaign coverage. Apart from the horse-race coverage there, I think the way they handled the email stuff, and the Comey letter in particular, was hard to defend. I think the way they minimized the Russian investigation is hard to defend. I think their post-election handling of it … Trump keeps saying they apologized. They didn’t. I think they should have apologized and actually gone back and re-examined all these different aspects of their coverage in the same way that they did with respect to the Iraq war.
With that said, I think that the Times is the best newsroom in the world overall, or at least one of the two or three best. And so I very much appreciate its coverage of the White House. I think their midterm coverage—I haven’t had too many critiques of it necessarily. But they really, really badly screwed up 2016 and I think they screwed up after 2016 by not owning up to the screw-up. But you know, still it’s the best news organization in the world. Although the Post would have a good argument.
Did you feel that the type of journalism you wanted to do while you were there was not welcomed by the people who determine politics coverage?
The Times is a place with a lot of different fiefdoms, and I think there were lots of people who were big supporters of FiveThirtyEight. It was never really integrated. There were literally some incidents where I remember when we were in the New Hampshire primary in 2012, I guess it was, where the FiveThirtyEight people come into the room in whichever hotel where all the journalists are huddled and it’s like a little bit of a high school kind of lunchroom cafeteria setting, where it’s people don’t want to talk to the—whatever you want to call us—data journalists or whatever.
And so there were lots of people at all levels of the Times who were supportive of FiveThirtyEight, but there was definitely a rivalry. I would say definitely the traditional reporters were—with some but not many exceptions—not all that helpful. And in some ways, that’s fine. We got along great with the graphics folks. We got along well with the sports desk. And there were, again, lots of rank-and-file people at the Times who really liked what FiveThirtyEight does. There was support from some top editors. But it was never … there was never much hope of integration of FiveThirtyEight.
Democrats are obviously going to nominate someone in a little under two years, and I think the debate about that is going to be: Are they going to nominate someone who is, broadly speaking, a centrist to appeal to middle American voters, the Obama-to-Trump voters, or are they going to nominate someone who’s going to excite the base the way we’ve seen with some of these candidates who have won by running more left-wing or liberal campaigns, like Andrew Gillum, the Democratic nominee for the governor of Florida?
When you look at the electorate, the Democratic electorate, and then the electorate that will vote in 2020, which is everybody, how much do you think that dichotomy is helpful to think about, and how much do you think it’s not helpful and it’s a simplified narrative of how we look at primary choices?
I think there are really three roughly equal parts of the Democratic electorate, one of which are white progressives or white leftists, you might call them. I’m careful about how I’m using the term liberal these days. So white progressives, let’s call it. Then you have white establishment, or moderate, voters. And then you have nonwhite voters. And roughly speaking, it’s about a third, a third, a third as far as the overall share of the primary electorate. And you have to win two out of those three groups to win the nomination, roughly.
So I do think that sometimes it’s easy for reporters to forget about the nonwhite voters in the electorate and the importance in the primary of the black vote and the Hispanic vote. Maybe that explains the success of someone like Ocasio-Cortez, who did very well both with white progressives and with Hispanic and black voters in New York. And so that’s where I think sometimes the analysis becomes a bit limiting.
But for sure, I think the ideological divisions of the Democratic Party are a lot clearer than they used to be. It certainly seems like there is a lot of energy on the left. I think one of the most remarkable stats from 2016 is that big, gigantic split between how people under 30 voted and people over 30 voted. Where it was literally something like Hillary won over-30s by like 70–30 and Bernie won under-30s by 70–30. It’s very rare to see an age split that sharp in a party. And you would certainly assume that you would rather be the rising, up-and-coming population than the older, aging-out population. And so that would seem to foretell the left being very viable.
With that said, you could have a situation where you have six or seven or eight progressive candidates and two or three establishment candidates, and so that could actually just—by the way the numbers work out—put things on a more level playing field potentially. I do think when you have a playing field this large, one of the lessons we learned in 2016 … and by the way, if you were listening to me and saying, “Oh, these guys are proud of how they did in 2016,” we’re proud of our general election coverage. We thought our coverage of the GOP primary was pretty awful. But there is kind of literally an element of chaos, I think, when you have eight or 10 or 12 or 15 candidates running. And it’s kind of like trying to predict where the four ball will end up when you have a break on the pool table. It’s literally a little bit chaotic.
