Medical Examiner

Data Can’t Decide When the Pandemic Is Over

We might have to do that ourselves.

A marquee on the theater reads "One Year Closed 3/11/20 to 3/11/21"
The Paramount Theatre in Seattle on Monday. Karen Ducey/Getty Images

Subscribe to What Next on Apple Podcasts for the full episode.

Early in the pandemic, the COVID Tracking Project, a volunteer effort led by journalists at the Atlantic, emerged as the most reliable source of state COVID data, from testing and infection numbers to hospitalizations and deaths. One year later, on March 7, the project shut down. It was always intended to be a stopgap measure, to prod the government into acting, and its founders now say they’re “satisfied” with the data the federal government is producing. But that doesn’t mean the data we have now is perfect, or that the pandemic is over. On Wednesday’s episode of What Next, I spoke to one of the project’s co-founders, Alexis Madrigal, about the lingering gaps in COVID data and what it will take to truly end the pandemic. Our conversation has been condensed and edited for clarity.

Advertisement
Advertisement
Advertisement

Mary Harris: You wrote, “President Donald Trump’s incompetence slowed the pandemic response, but did not define it. We have learned that the country’s systems largely worked as designed.” Could you explain that? Why do you think it revealed something more global?

Alexis Madrigal: I think the easiest answer to that question is: Did we have a data system for tracking tests? No. Did we have a data system for tracking hospitalizations? No. Did we have a high-speed system for understanding deaths? No. I think that a lot of public health officials, stunned as they are by what has happened, cannot believe that the system that they had been working on—there’s all these pandemic preparedness plans stretching back decades—that it wasn’t enough. Things like the nonpharmaceutical interventions, social distancing, all those things—those were pandemic preparedness plans and those work just fine. But the assumption that went into those things would be that there would be data that would be useful for decision-making.

Advertisement

That would just exist.

That would exist. That it would be there. That we’d know the number of cases, for example. But we didn’t. Not even close. We knew what we could confirm, but we still to this day have no idea how many people were infected. Maybe 80 million, maybe it’s 100 million, maybe it’s 130 million. That’s pretty wide error bars there.

Trump certainly made things worse. Absolutely. There’s just no doubt about that. But the systems that would have needed to exist before the pandemic just did not. And so the systems that got built, they’re emergency response systems, not something that pandemic preparedness had actually gotten ready ahead of time.

Advertisement

Why do you say the system was designed that way versus some things were overlooked?

If you look at the depth of pandemic preparedness planning, a big chunk of it functioned on this model of, well, we’re just going to tell you what needs to happen, and then you all out there will do it. You’ll listen to these public health officials. And they weren’t really planning on the level of data sharing that was really necessary. I mean, part of the design of the system was not to provide a lot of this information to the public. That wasn’t really how people thought about what would be needed to give to the public. There’s been a ton of thought about what we call tactical communication, which is what do I need to tell you to get you to do something and what data do I need to provide to the public to build trust with them to maintain over this incredibly long period of time, these super painful viral countermeasures. Trump made that worse, by taking the best CDC communicators out of commission. But the communications playbook that they were planning to run was going to be tactical in that way. It would have used the preexisting CDC data systems, which, again, did not have all of the components that we would have wanted and needed.

Advertisement
Advertisement

Another data issue is baked right into the country’s DNA. The share of power given to state governments has meant that having sound data at the federal level depends on getting good information from each state—and having good communication with them, too.

Federalism is what it is. The states have a lot of power to decide how things go within their borders, and not just what the governors do, but even what data is available is really up to governors. And we know that governors pay very close attention to the dashboards that are out there for their states for COVID, and that that allows them to shape the narrative around what’s happening in the state. That doesn’t necessarily mean that they’re covering stuff up. It just means that they choose what information to communicate at the state level. That is just part of how the American system works, and I think that really shows in the variety of outcomes that different states had. It’s a little hard to blame the states that got hit really hard in March and April because no one really knew what the hell was going on, and we couldn’t test anybody, so we have no idea how many cases, and if you can’t count cases, you end up counting bodies. And that’s what happened to a lot of the Northeastern states. But later on, people did know what was going on and still had these very different approaches to trying to protect people’s health.

Advertisement
Advertisement

My impression is that even the data that the states or the federal government is releasing now is far from perfect. For example, COVID deaths are reported by the CDC sometimes a month after they happen, and that means that when you’re looking at a chart of COVID deaths, it’s hard to know what it means.

Yeah, that’s exactly right. The death accounting seems like it would be simple, right? When somebody dies, that’s obviously just a fact. But each state does it differently. There’s different components of every death certificate, and COVID could go in different places. And one thing that we’ve seen happen over and over and over again is that when a place gets hit particularly hard, its infrastructure for handling those death certificates gets overwhelmed. So what you see pretty typically now is there’s one peak that is pretty close to the peak for hospitalizations, and that’s deaths that for whatever reason got reported very cleanly and made it into state and federal data, let’s say, within a week. And then there’s a whole other set of death certificates that get backlogged and that end up some sort of weeks later—Virginia did this in a very intense way, Ohio did this in a very intense way—suddenly, long after the peak has passed, weeks or a month afterwards, you start getting huge numbers of deaths reported by the state. And the reason that this is actually quite a bad thing is that there’s at least some research that the thing that really changes people’s behaviors in terms of being safer is when they see death reports, that death numbers can actually drive behavior change. And so if we knew about those deaths earlier, maybe people would be safer en masse and maybe fewer people would die.

Advertisement
Advertisement

I think the other thing that just remains a huge problem is testing data. Testing data is important on its own in a sense. But a lot of states actually use testing data in conjunction with cases to create a positivity rate. And then they use that positivity rate as a gating threshold for various things. The problem is that it’s not clear how accurate it is to use that calculation. Some tests get reported in electronically very quickly, and other tests take longer. They don’t make it through the normal pipelines.

And this positivity rate data is being used to determine stuff like whether schools open, right?

Exactly, exactly.

When do you think you’ll know that the pandemic is ending?

Advertisement

I think it’ll be when the hospitalizations from COVID really, really drop off. And we’re expecting to see that because so many of the older folks who were so at risk for severe COVID are getting vaccinated. … It’s going to take a little while still.

I actually think the Biden administration setting Fourth of July as a “freedom from COVID shall ring” kind of day seems pretty realistic to me. And it makes sense. Hopefully this won’t all happen in this way where suddenly everybody’s doing everything. It would be a good idea, given these variants of concern, to turn the dial slowly and be willing to hang on for a little bit longer. That’s what I’m hearing from public health officials, and that really makes sense to me, especially given that there are some places in the data that are a little worrisome, where we’re seeing plateaus in cases and hospitalizations before we actually wanted. We wanted the numbers to go down a lot more before we saw numbers in the Midwest and the Northeast start to really plateau.

Advertisement
Advertisement

When you mention July or August, the summer, I just think, why aren’t we worrying more about a fall surge?

I am. I am worried about that. Clearly seasonality is a thing that’s real for this virus, and it’s real for other coronaviruses and all those kinds of things. What I hope is that by the fall, let’s call it October, we basically have everybody vaccinated, and it certainly seems like it’s going to be possible to do that in terms of supply. And then I hope that there are boosters ready for people if we get some of these other variants that seem to evade vaccination or natural immune response. So, yeah, I think it’s going to be worrisome, and I think there are very real trade-offs with the public health interventions that have been in place for so long. If our whole country is vaccinated, we get really vast numbers of people vaccinated, I think we have to say, well, we kind of did the best we could here, you know? It seems very difficult to maintain multiple years, particularly if deaths do stay at lower levels for a sustained period of time.

You talked to some experts who talked about this flu test idea, that if the COVID deaths got to about where the death levels are for flu at its peak, like 100, 150 deaths a day, that maybe that would be acceptable to people. And I thought, The flu kills a lot of people every year, and I guess we’ve decided we’re OK with it. But to layer another virus on top of the flu that is killing people at a similar rate could be a nightmare.

Yeah, it’s very hard because it’s not like there’s somebody out there who decrees, yes, this is acceptable and this is not. I think the scenario that people are probably not considering enough is that COVID gets 85 percent of the way better, but then just doesn’t from there. What happens? What would happen if there’s like 600 deaths a day for a long, long, long, long, long time, given that there are real trade-offs, mental health, economic, and otherwise? How long are people willing to sustain particularly the social distancing aspects of the interventions?

Advertisement

This is not a professional opinion, it’s a personal one: I see everyone around me just, like, falling over. Everything has been kind of a state of emergency, but you can’t actually sustain that forever. And so if we have to find a middle path to keep transmission low, or even if we can’t get to zero or near zero, then what? That’s the thing that’s been haunting me. I think things are going to get way, way better even from where they are now, particularly in terms of a hospitalization and deaths perspective, because of the vaccines. But what if we don’t get all the way? Then what do we do as a society? That, to me, seems like a very hard question.

Subscribe to What Next on Apple Podcasts

Get more news from Mary Harris every weekday.

Advertisement