The Limits of Coronavirus Predictions

Listen to this episode

S1: There’s this moment that I keep thinking about. It was on March 21st at the White House. The coronavirus task force was holding your briefing.

S2: Thank you, Mr. President. If I could have the first slide, please.

S3: Dr. Deborah Burks was standing at the podium and behind her there were these charts, charts that showed two curves. One was a steep bell curve and dark blue. And she called that a mountain.

S2: I think, you know, from that large blue mountain that you can see behind me. And I just want to thank the five or six international and domestic modellers from Harvard, from Columbia, from Northeastern, from Imperial, who helped us tremendously.

Advertisement

S1: This was the first time the White House had showed the public models, statistical projections of what covered 19 could do if left unchecked. That was the mountain, some 1 to 2 million deaths in the U.S. versus what might potentially happen if Americans took some measures to slow the virus spread. That was the smaller curve, a more gradual hill.

S2: It was their models that created the ability to see with these mitigations could do how steeply they could depress the curve.

S4: March thirty first I was like four or five years ago, right? So I think I basically remember it.

S1: This is Jordan Ellenberg, a mathematician and professor at the University of Wisconsin.

Advertisement

S4: That was probably the first moment when you heard the government of the United States really talking in this language to the people of the country as a way to think about what we can say about what’s to come.

Advertisement
Advertisement

S1: Dr. in the pandemic task force had presented two scenarios, a worst case and a better case. But to predict anything in between, that’s when the real work starts.

S4: What the models are missing is that it’s very hard to model human behavior, even if the aggregate it’s pretty hard to capture what people are going to do by some kind of differential equation or some kind of fancy network model or whatever it may be. So a lot of what happens over the weeks and months to come depends on how people respond to the epidemic in their own community and how people respond to the mandates given to them by their government.

Advertisement

S1: Ever since Dr. Bourke’s presented these charts to the American public, they seem to be everywhere, not just in the media.

S5: As we try to wrap our heads around when this will be over and how bad it will be. But in the highest levels of government, as they make policy decisions that are bringing economies to a halt, upending lives and costing jobs. So today on the show, we explore how these models work, what they’re good at showing us and how they can impact your behavior and mine.

S6: I’m Lizzie O’Leary, and this is What Next TBD, a show about technology, power and how the future will be determined. Stay with us.

Advertisement

S1: When you model how a disease might spread, you start with a few seemingly simple inputs. The infection rate, the fatality rate, the number of people who might be susceptible. But of course, there were so many other factors time, community action, all the sorts of things that governments want to analyze. So why do we need these things in the first place? You know, what is the utility of having a model or several models? When we’re faced with a pandemic?

Advertisement

S4: Well, it’s some very basic level. Of course, we want to know what’s going to happen. But I want to emphasize that the easiest question to answer and this is true and that this industry in physics, it’s true in any quantitative or scientific context is what’s going to happen if nothing changes, if everything goes along exactly how it has been going. Well, we have some data about what happens when everything goes along without taking serious suppression back. So we can use that data to extrapolate and see what would happen in the future.

Advertisement

S1: Jordan says that modeling how the virus spreads without containment measures in place was a crucial first step in mobilizing governments to take action. A team from Imperial College in the U.K. led that effort.

S4: The Imperial College model was the first one to get really widespread public uptake. I think partly because it was taken seriously by the British government and those numbers were extremely scary. What they were modelling is what happens with this disease without large scale serious attempts to suppress. And it’s almost hard to remember now. But if you go back one month, people were still saying, is this just going to pass? Like I was on a plane like less than a month ago. Like lots of people were. The airports were full. Schools were open. And I think people were asking, are we going to need to take any kind of serious measures? And so I think that model was an attempt to track what would happen if things go on as they have been.

Advertisement

S1: What they predicted was as many as five hundred ten thousand deaths in the UK and 2.2 million in the US. And this model has been credited with encouraging governments of both countries to enact much stronger measures in response, which led the modeling team to a more complicated question.

Advertisement
Advertisement

S4: They started to say, well, what happens under various models of suppression? And that’s a lot harder to do because then you’re not trying to say, let’s extrapolate from data we’ve already seen. They’re saying, let’s hypothesize some change that we undertake in what will be the result. That’s an inherently harder problem.

S7: You know, when you try to predict what a virus is going to do. We can kind of learn pretty quickly how a virus behaves when you’re trying to predict what people are going to do. That’s a lot harder than a different way.

Advertisement

S8: Can you give me a sense of the main models that get talked about? You mentioned the Imperial College. What these models are and how they differ.

S9: The model for Imperial College.

S4: What they are doing. It’s very traditional, as what’s called sort of a differential equation model. Basically what they’re doing is saying at any given moment, we make our best estimate as to how many people are infected with this virus, how many people are recovered, and thus we think have at least short term immunity. Even that we don’t really know. Right. And that’s an important remitters model, too. And then how many people are still susceptible? That’s the reservoir from which the virus drops, people who were still walking around ready to be infected by the disease. And then it kind of goes day by day. Okay. You predict each day each infected person will, in fact, an average of so many more people out of the people who are still susceptible. Also, each day, some percentage of the people who are infected will recover and sort of move into the bucket. The people who are recovered. And by doing this, sort of you can estimate, given the data today, what we predict about tomorrow. Now plug it in to get given the data tomorrow. What can we predict about what it’s going to look like the day after tomorrow?

Advertisement
Advertisement

S1: And then we’ve got this model from Seattle, from the Institute of Health Metrics and Evaluation. What’s their approach?

S4: Their approach is totally different because they’re not trying to model the spread of the disease. Step by step, they’re saying, let’s look at one place where we’ve already seen the disease in some sense run its course. Other interventions, such as China, sort of look at what the shape of the curve looks like and then posit that the curve of deaths is going to go up and go down in a certain shape. Might be higher, might be lower, might shift at makes it to the right. But the shape they say, let’s hypothesis that that’s the shape. And then they do what’s called curve fitting. They take the data that we’ve already seen and say of all the possible curves of roughly that shape, which one fits the existing data best. And then they hypothesize that that’s what’s going to happen in the future.

Advertisement

S1: Last week, the model predicted some ninety four thousand deaths in the US by late summer. And now we’ve got a new estimate of roughly sixty thousand deaths by August. What’s your reaction to that kind of change?

S4: This is so important for people to keep in mind. The Seattle group is doing a very good job of putting very wide ranges around their number. But, you know, typically when they say, OK, in this state, we think we’re going to see two thousand deaths. They’ll also give your rate. Well, maybe it’s 300, maybe it’s ten thousand. And so we should feel good about those numbers going down. We should also recognize that they have been telling us from the beginning that it’s really not possible to make an estimate down to the single person or this or 10 people or 100 people or probably even a thousand people. Given the smallness of the data we have right now, it’s something to be conditioned by the age of big data to think of machine learning prediction. That’s really, really good. This is not big data we’re talking about. This is extremely small, but that’s what we have to work with.

Advertisement
Advertisement
Advertisement

S1: When you talk about the specificity of those predictions, I think one thing people at home are watching is when’s the peak, when’s the worst part? And that model has sometimes given us actual dates. And I wonder, is that is that maybe something we’re fixating on too much?

S4: Yet you’re wondering exactly the right thing to want. I wish they would do is when they were reported the peak, their estimated peak, that they would give the same kind of large error range because the uncertainty of the parameters in their model means that their estimate for when inspections are going to peak is no more certain than their estimates for the total number of deaths. The total number of cases and I worry that when they say, OK, in this state, it’s going to be April 22nd and other data can be made for people like putting that on their calendars. That’s a mistake. That’s a mistake. I think those should be thought of as saying, OK, we’re saying it’s April. It’s not going to be June.

S1: Do you think there’s a point in which a model can tell us? All right. It’s safe to go back to work and you can stop social distancing now.

S4: So that brings us to a really interesting question, which is that another critique that I think despite valid that has been made about the model is they are really focusing on what happens in the next three or four months. Like let’s say that we sort of maintain some level of suppression, whether at this level or a different level from now through June or July or August, I think is what you use. You also got to ask what happens after that?

Advertisement
Advertisement

S10: We’re very likely not going to have a vaccine, but nobody seems to think that’s realistic possibility, which means that when we do open things up, we’re going to be back to that model where the virus is able to move unimpeded and then maybe we see a peak that’s just as bad as the unimpeded except later instead of sooner.

S4: So the point of all this is the later is much better than sooner, because ideally what we’re doing right now with all this pain we’re undergoing is to buy us time to buy hospitals, time to brace themselves and build up capacity to buy doctors, time to sort of test like a million different potential treatments to understand what can get people out of the ICU sooner or prevent them from having to go into the ICU in the first place. And it’s the biocides develop a robust system of wide scale testing that allows us to stamp out localized outbreaks before they get big.

S1: There are signs, very, very early signs that these efforts to buy time are working and the disease models have reflected this change, predicting much lower mortality rates. What if someone who looks at this and says, well, I don’t know. They said it was gonna be bad, but now the model looks better. So maybe this isn’t as big a deal as the experts or the government was saying?

S4: Well, I do think that’s a real danger. And I actually do think you saw a little bit of messaging that sounded like that coming out of the White House last week. And I think to their credit base to sort of quickly backpedaled from that and made it clear that what is happening and you see those numbers go down and when we see the diarists of those projections start to fade away, that is because of what we’re doing. So when you see those numbers material go from 2 million down to 60000, that’s not those research reports. And you would see me and the researchers say good job conditions have changed. And when conditions change, the model has to change with it.

Advertisement
Advertisement

S1: You know, I think a lot of us who did not go on to study graduate level math think of it as not a living thing.

S4: Absolutely. I mean, look, what I always say about math is that it’s not some kind of foreign thing that a computer invented. It’s us. It’s our human way of thinking. Just made a little bit more formal. So you asked the person are looking at the news and looking at what’s happening in New York and looking what’s happening in Louisiana. You’re looking at both the bad things and the good things, and you’re constantly revising your estimation about what’s happening with the city in different directions. Right. If you do that with a differential equation or you do that by fitting a curve to some set of points that you have in your computer, you are actually doing the same thing, the same mental process, just in a more formal way. But it’s not fundamentally different from the way that anybody would think about it as these. All of us are constantly revising our sense of what is to come and our sense of how what is to come depends on the decisions that we make.

S1: What would you say to the layperson who. He’s trying to understand what role this model should play in their life for their understanding of a virus.

S4: This is all about aggregates. So good example, I think this whole controversy over mass. I think first a lot of people were like, well, that’s not going to reliably keep me from getting this disease. I’m out in public. So what’s the point? And it’s true that that’s not going to reliably keep you from getting the disease. That’s a fact. No one disputes that, especially if your math is sort of super fancy. And ninety five maps that we should be reading for health care providers anyway. But if you think of the aggregate, maybe wearing that mask is going to reduce by 20 percent, 30 percent, 50 percent. Who knows the average number of people that you, if you happen to unknowingly be infected, are going to transmit the virus, too.

Advertisement
Advertisement

S11: So I think what math would ask people to do is, you know, we’re very individualistic in this country and in many ways that’s good. But at the moment, we kind of have to think of ourselves as each of us as one part of a very large average. And how are we going to change aggregate behavior in a way that overall reduces the total amount of transmission, even if it doesn’t affect our own personal health that much?

S12: Jordan Ellenberg, thank you so much. Thanks for having me on.

S13: Jordan Ellenberg is a math professor at the University of Wisconsin, Madison. All right. That’s our show for today. What next? TBD is produced by Ethan Brooks and hosted by me. Lizzie O’Leary. And it’s part of the larger what next family. TBD is also part of Future Tense, a partnership of Slate, Arizona State University and New America. And I cannot recommend enough that you go back and play Monday’s episode of What Next? It’s about Coby, 19, on Rikers Island, and it’s essential listening. What next? We’ll be back on Monday. Thanks for listening. Talk to you next week.