Why the Vaccine Websites Suck

Listen to this episode

S1: Late last week, I was on the Internet losing my mind, I was trying to get my mom covid vaccine appointment in New York State, but the website kept crashing, showing error messages. I’d see an open appointment. And as I clicked on it, it would disappear. After about five hours of frustration and a little yelling, I saw this one woman on Twitter who said she was having success booking appointments. Later, I asked her what her secret was. I noticed in your email to me, you said you were doing a lot of this in incognito mode because you’d learned this trick before. How did you learn that?

Advertisement

S2: This is super dorky, but we it’s totally from booking Disney vacations for my family.

S1: That’s Jessica Allen. She was looking for appointments farther north in New York State than I was, but she had cracked the system using her Disney booking knowledge.

S2: It used to be that you had to book your dining reservation six months before your trip and you had to be on at a specific time in the morning in the dining, reservations would open up and like things are very competitive, just being in incognito mode for whatever reason, like it helps give you access to really what’s available.

S1: Here’s what it’s like to try to get a vaccine appointment in New York State right now. First, you hit a screen to confirm the person is old enough for a frontline worker, that sort of thing. Then you end up on a page with links to various locations that might have the vaccine, either state run sites or private pharmacies. You pick one and only then can you try to make the actual appointment. And that’s assuming your browser hasn’t crashed or that the location hasn’t run out of the vaccine. In the meantime, I failed many, many times, but Jessica got really good at it and she figured, why stop with her parents? She kept the websites open, then called up other family members to book them their slots conferenced in seniors who weren’t great at using the Internet and walk them through the screening questions. I had a lot of like 80 year old ladies giggling when I asked them if they were pregnant, you know, she reached out to her neighbors at one point early and Friday, I remembered my neighbor across the road, a school bus driver.

Advertisement
Advertisement
Advertisement

S2: And so he qualified. And so I called him. I said, hey, I feel bad that I didn’t think about this sooner, but are you trying to get an appointment? If if you are, can I help you if you need help? And so I was able to book him right then on the phone and then he called two other people that he knew qualified and told me later that they were both able to get appointments.

S1: All told, how many people do you think you have helped get appointments for? 50. It’s been 50.

S3: Yeah, because of Jessica, 50 people are going to get vaccinated. Seniors, people who aren’t Internet savvy, people who live in rural areas without good broadband. And that’s great. But the fact that it took this effort and the fact that people who don’t have a Jessica in their lives are at a real disadvantage, that is very much not great. Today on the show, why it’s so hard for so many people to sign up for vaccines, why the online systems are failing and how we could make them better at this crucial moment. I’m Lizzie O’Leary, and you’re listening to What Next, TBD, a show about technology, power and how the future will be determined. Stick with us.

Advertisement

S1: Over the course of the past year, a state struggled to get covered data out to the public to update their websites or to schedule testing. There was one group that seemed to be everywhere.

S4: If you put yourself in the shoes of a non-technical decision maker in like a local government, you’ve been handed this urgent priority and you need to make the tech work.

S1: That’s Rafael Lee who directs the health program at UC Digital Response, a collection of volunteers, engineers, user experience designers and technologists who’ve been offering help for free to overwhelmed state governments.

Advertisement

S4: So some state local governments already have technical teams that have the ability to design new services from scratch or adapt open source review vendors. They’re not usually very big teams. Most state governments don’t have these teams and so they often go to the private sector for solutions. And software engineering is this crazy, topsy turvy world where the things that seem really hard are sometimes pretty easy and the things that seem really easy can be pretty hard. So unless you have senior technologists on staff who can help you review your vendors before you sign the contract and pay them money, you’re just going to buy based on the brochure that they give you and how our state and local governments reacting to the problems with vaccine rollout so far.

Advertisement

S1: I mean, are they frustrated? Is that something that you’re hearing?

S4: I think mostly people are confused. Hmm. It’s a really hard problem to solve. It’s the largest vaccination rollout in modern history. And the reason it’s so complex is primarily because states are following a phased approach that prioritizes different groups of people. But they don’t always know exactly how many doses they’re getting or when they’ll arrive. And when these things aren’t certain, it’s hard to plan ahead. More recently, we’re seeing that the eligibility standards and the enforcement of those standards are changing as states adapt to the situation. And that makes all the technology projects a moving target for the health agencies.

Advertisement

S1: So I guess, yeah, as the standards change, you know, every week, every couple of days, it’s pretty hard to be nimble and respond to that. Exactly. You know, I think from the perspective of people trying to schedule vaccine appointments for themselves or their parents, there’s a lot of frustration. And it’s a sort of overwhelming, singular question. How could they not get this right? There have been months to prepare a lot of systems in place to test this stuff out. What is happening behind the scenes that is is explaining some of these issues.

Advertisement
Advertisement

S4: So every state and county has different needs, but they all face pretty similar challenges and they’re using technology to address other problems. So there’s the part that the public sees and then there’s the part that we don’t really see. So let’s start with the part that we do see. First, there are the informational websites that tell people when they’re eligible for vaccines and where they should go. The CDC has shared guidelines and recommendations about the order in which vulnerable groups of people should be prioritized. But ultimately, each state has the discretion to make its own plan.

Advertisement

S1: And these plans are usually complex enough that they’re hard to communicate succinctly in plain words, which which sort of makes me wonder why do this online at all or why do this, particularly if you’re thinking about seniors and marginalized populations, why not meet them where they are as opposed to sending them to a kind of janky website?

S4: Ideally, you have an overall effort that is able to connect to all your populations. There should be an offline and an online component. The offline components don’t scale very well at all. So, for example, if you had scheduling that was only based on the phone, you’d have everybody calling in all the time to the phone lines and you’d have to scale up the people to handle those phone calls. And that is a very expensive and laborious and tedious operation. So if you have this kind of split response where the people who can get online can go to a website and do it there, then that frees up the phone lines and reduces demand on city hall or the vaccine providers to handle the people who can only get in by telephone, by phone.

Advertisement
Advertisement

S1: And there are other ways technology should be able to help the process of getting vaccines into people’s arms.

S4: There’s also things like reminders. So the Fizer and malaria vaccines are multi dose vaccines and people are going to forget to come back for their second dose unless they get a reminder. I read a study in the UK about the hepatitis A vaccine. In the study, only 11 percent of patients completed their two dose regimen. Wow. So if we send out some reminders, that will definitely help.

S1: But reminders and websites and apps and all the public facing technology are still only part of the effort. There are complex logistics and lots of health data that need to be managed on the back end.

Advertisement

S4: There’s tons of things that we don’t see. So there’s things like vaccine inventory management. The states and local health jurisdictions have to track and manage inventory, by which I mean they need to know where the doses are and which sites are running low. They have to keep track of who has gotten which vaccine. They’re not interchangeable. So if you start with Pfizer, you have to finish with Pfizer and you can’t. Substitute men enough for the second dose. They have to collect patient information ahead of time to assess eligibility. They need to find places to even have vaccination sites. So in some parts of the country, governments are planning temporary pop up, even drivethrough vaccination clinics in addition to the hospitals and pharmacies. And most importantly, they need staff. So you can have all the doses in the world that your vaccination site. But unless you have trained licensed medical staff to put shots and arms, people are not going to get their doses.

Advertisement
Advertisement
Advertisement

S1: All of these things make so much sense what you’re saying. And yet I think, wait, why are we having this conversation in mid-January? These don’t seem like things that are impossible to anticipate.

S4: You’re right. They’re not impossible to anticipate. I think one issue that has plagued the state and local governments that we work with is that it’s been difficult to get a clear picture of who’s making this decision. And in the absence of a unified plan from the federal governments, sometimes it isn’t exactly clear who is responsible for what.

S1: I wonder if you can tell me if states and localities are are hopeful that in a new administration, they might have more clear direction from the federal government?

Advertisement

S4: I think there is a spirit of cautious optimism. So when President Biden unveiled the American rescue plan, civic technologists discovered that it includes more than 10 billion dollars of funding for cybersecurity and information technology. A lot of people are interpreting this as a very positive sign that the new administration is thinking about delivery as well as policy, and they’re willing to put their money where their mouth is and hire more technology experts in the government. So people are hopeful. We’ll see what it looks like in practice.

S1: I think about this moment that we are in. It is. For so many people, hopeful, but I also have heard the incoming director of the CDC warn that another 100000 people could die and we are in this terrifying winter. I guess, I wonder, do you feel hopeful that the scale of users accessing that vaccine information so that they can get vaccines is improving?

S4: I am hopeful. And the reason I’m hopeful is because I saw the same thing happening when governments were setting up testing sites. Hmm. In the beginning, it was pretty confused. There weren’t that many good solutions out there. And governments were trying really hard to figure out who do we partner with, how do we get enough of the materials to even do the tests? How do we partner up with labs? How do we handle the same issues, including scheduling and reminders that we’re facing now? And it’s not perfect now, but we’ve come a really long way.

Advertisement
Advertisement

S1: State governments have made big strides with the help of groups like Rafael’s, and many have had the benefit of time to work out kinks in their testing systems. But still, why is the government relying on volunteer groups to deliver essential services? Yes, this is a huge and unprecedented crisis, but the roots of these tech problems run much deeper than that. After the break, why the government is so bad at tech anyway. Stick with us. At the same time that I was trying to get my mother a vaccine appointment in the New York state system, Hannah Chank was trying to do the same thing for her mom in a different one, the system for New York City.

S5: My mother told me that she had found somebody next door on the app who was going to help her get registered. And I was like, you don’t have to do that. I am a I work in technology. I will do it for you. And I’m thinking this is going to take life even with all of the stuff happening. I thought, well, how hard could it be? Like I could build a page like this shortly. I can figure it out.

S1: Hannah is a public interest technologist at New America. She was part of the U.S. Digital Service and the Department of Homeland Security in the Obama administration. And she’s worked on user experience design for more than a decade. And even she couldn’t figure out how to sign her mom up on the city website.

S6: At the same time, we’re getting all of these things like tweets, like, oh, not enough people are signing up and vaccines are going to spoil and things are being defrosted in.

Advertisement
Advertisement

S5: And I was like, but we’re here in Brooklyn trying to get the appointment with this very nice seventy seven year old lady and we can’t do it.

S6: Ever since they announced that there was going to be a vaccine rolling out, I started saying, what’s going to happen? How is the scheduling going to work, the logistics going to work, how this is going to work from a tech perspective. And then just to find myself like in the middle of that with my mom. Yeah, pretty much my head exploded.

S1: I’ve been through this, too, with both of my parents. And I guess I wonder for you, for someone who has built a career around public interest technology, you’ve worked inside of government tech teams. This must land very differently for you than it does for someone else.

S5: I think that for me, in a weird way, it feels personal in part because, like I know like I I literally know the very people who work for the mayor’s office who could very easily fix the whole situation. There are a bunch of tech teams in New York and they have not been asked to work on this.

S6: We have been preaching the public interest technology gospel for about four years now at New America and saying, like, it’s really bad to not have technologists in government for a whole bunch of reasons. You don’t really want to see all of those things come true. You know, like, yes, we’re saying this is going to be a problem when it actually materializes. It’s almost it just feels terrible.

S1: I asked Hunter to back up to tell me the story of the government’s relationship with digital technology.

Advertisement
Advertisement

S5: I think the story really starts in the 90s where all of a sudden the private sector started to realize, like, oh, we actually need a functional website where people can communicate with us and maybe even like order things and check their order. And maybe that’s like our primary mode of communication, even as people like chat with us.

S7: None of that happened in the government in part because of budget cuts. And they didn’t have the staff, but also government did not see its role as being customer centric. A lot of the mindset in this country is that government will help you if you really need it, but you better really need it and you’ve got to prove how you really need it.

S1: Does that differ in other countries? I mean, how do we compare to some place else that that is also, you know, maybe trying to get people vaccinated or just provide services online?

S7: It is really different from other countries. We had some conversations right around when the Kahrizak was rolling out and we compared notes with the deputy German finance minister because in Germany they had gotten the checks out incredibly fast. And so we want to know how they did that. All of the language is, don’t worry, government is going to be there for you. We’re going to help you. We’re going to get you the money that you need. Just tell us what you need and we will provide it.

S1: Over the last year, there have been a lot of stories about outdated government tech infrastructure that one of the most famous being New Jersey’s unemployment system, written in COBOL programming language from the 60s. When you look at this, how much of the problem do you place at the feet of outdated technology and how much is on a lack of focus about what users need and want?

Advertisement
Advertisement

S7: The outdated technology is really more a symptom than the main issue. It’s it’s the result of not paying attention to keeping tech updated because you don’t think it’s important, because you don’t see it as being integral to your mission. If we’re talking about New Jersey, nobody is sitting there saying, you know, this is going to fall apart because it’s written in a language from the 60s or, you know, maybe we might want to think about investing in making this more customer centric. That’s that’s not part of the language. I don’t want to say anathema, but it is just contrary to how the policy level people think about their role.

S1: Well, how do you make that connection then? When I think about sort of who is tasked with delivering on what the policy people come up with and how it gets from a, you know, digital standpoint to the customer who makes that happen.

S7: So this is, I think, why we are finally getting these conversations about user experience and people are finally paying attention to it because there has not really been a connection in the past. The policy people make the decisions and then the doers implement it. Today, the people on the doing side are like, how? Well, how how do we do that? Because they’re not they’re not equipped.

S1: Some people are going to listen to this interview. And in fact, I’ve seen this reaction on the Internet a couple of times and say just bring in the private sector. Is that a good answer?

S6: That is a terrible answer, really. The private sector is already in. If you think about the fact that all of this work is done by contractors, it’s the same ten people over and over and over again. It’s Deloitte, it’s IBM, it’s Oracle. It’s all of these big tech consultancies they’re already in. They are not looking out for people’s best interest. They are typically looking for how can we make money? How can we write ourselves into an ongoing contract? For example, we’ll license you a piece of technology. We won’t build it for you.

Advertisement
Advertisement

S7: And then you have to pay us licensing fees to maintain this very quickly outdated piece of technology. It’s, of course, a very natural response to say, well, just get Eventbrite or like surely there is scheduling software that someone could use.

S1: Right? This feels like looking for concert tickets.

S5: Right.

S7: Or like any of the other things that we do a million times, like booking on Airbnb or like any other interface where there is a map and you pick a location, then you schedule a thing like we’ve all done that a thousand times, so why can’t it happen in the in government? And part of the issue is that what is happening behind the scenes is actually really complicated.

S8: So there’s a whole question of respecting people’s privacy, but also. Being able to collect the right data so that we know where the vaccine has been distributed, it’s more complicated than just throw the private sector at it. Hannah says that what a lot of users, more well-off users who are trying to make appointments are discovering is the experience of being poor, of needing government services that often just don’t work.

S1: In a story you wrote about this, you wrote that a lot of Americans really only have to interact with government occasionally doing their taxes, go to the DMV, maybe traffic court. But when we are talking about Americans who rely more heavily on government services, the failure of technology is a huge added burden in terms of time and effort required to navigate all of this.

S9: One hundred percent. I think it’s partially just that the world has kept evolving and government has not. But yes, I think an enormous piece of this is that the people who are frequent users of government services are people who are already marginalized or lower income. These are the people who need help. And so, yes, they’re used to it being really awful to interact with government.

Advertisement
Advertisement

S1: So in many ways, the fact that you and I are having this conversation is perhaps because more well-off Americans are noticing this right now because of the pandemic. Absolutely.

S9: Absolutely. The entire population all of a sudden is having this wake up call that everybody else who has been interacting with government for a long time, this can’t possibly come as a surprise to them.

S1: One of the things that I’m really struck by, and this is the failure or what seems like a failure to think about those most vulnerable populations and to think about they the way they might be trying to access these sign-up systems. For example, if your only Internet connection is on mobile and the mobile dropdown menus don’t work, you have no recourse.

S9: Guess that that is that is true. There are a significant number of government services that are designed purely for a laptop or don’t function well on mobile, even though we know from research that a large percentage of the population is actually going to access this via mobile and maybe even over a cellular network, because people don’t have laptops or people don’t have Wi-Fi.

S1: While states like New York and California are struggling to get people signed up and vaccinated, there are places that seem to have hit their stride. When I ask people on Twitter to send me their vaccine sign up stories, I got emails from people in Michigan who had easy experiences. And Hannah says things are looking pretty good actually in Alaska.

S6: In Anchorage, they built a tool that will go and look for appointments at all the different places and then deliver it in one interface. I don’t know if it’s because they have that and they they have a tech team and they are using it. But Alaska is doing really well. And interestingly, they told me that they also gave the tool to North Dakota. North Dakota is also doing really well. So that was wonderful to see that. However, they were able to make that happen. They’re using their tech talent, they’re using their innovation team, and they’re doing it in a way that is open source so that other states can benefit from.

Advertisement
Advertisement

S1: The Bush administration’s goal is 100 million vaccine doses by April. And I wonder when you think about that number and the distribution failures we have seen so far, how would you design a program to reach and enroll the largest number of people possible so that that 100 million doses could maybe happen?

S7: Part of what hasn’t happened is some real data analysis right now, we’ve been handling it as though all 50 states are equitable and all 50 states have the same needs. I don’t know that that’s true. At a bare minimum, just doing some analysis around where it makes sense to vaccinate would be a huge step forward.

S1: And then she had one other thought. Less tech related, more human.

S9: Why can’t we all sign up? This keeps this has been driving me crazy. Like I would like to know when I’m scheduled, even if it’s August, that would just give me peace of mind. And I as long as you know that an event is going to happen, you can start to plan your life after that. So many people are suffering from depression and all other all the other things that are coming with the pandemic. So if people could, at the very bare minimum, have a date when they’re like, well, at this date, I’m going to be vaccinated, I’ll be fully inoculated and I can restart life, I think that would change people’s attitude a lot.

S10: Honasan, thank you very much. Thank you so much. This was fun. Hannah Shank is the director of Strategy for Public Interest Technology at New America Rafaeli. She is the director of USDA’s Health Program. And Jessica Allen, who heard at the top of the show, lives in northern New York State. And hopefully, by the time you hear this, my mom will have gotten her first shot thanks to a state site Jessica flagged for me on Twitter. TBD is produced by Ethan Brooks and edited by Allison Benedikt and Tori Bosch. Alicia Montgomery is our executive producer. TBD is part of the larger What Next family. TBD is also part of Future Tense Partnership of Slate, Arizona State University and New America. Have a good weekend. Mary Harris will be back on Monday. I’m Lizzie O’Leary. Thanks for listening.