Slate has relationships with various online retailers. If you buy something through our links, Slate may earn an affiliate commission. We update links when possible, but note that deals can expire and all prices are subject to change. All prices were up to date at the time of publication.
This article is adapted from Power to the Public: The Promise of Public Interest Technology by Tara Dawson McGuinness and Hana Schank. © Princeton University Press 2021.
In 2005, the U.S. Citizenship and Immigration Service, the successor to the Immigration and Naturalization Service and the federal agency responsible for green cards and citizenship applications, began a project to digitize the nation’s immigration system. At that time the immigration system was largely paper-based: One of 94 different forms would be turned in by applicants at one location and then mailed around the country to different locations depending on form type. The goal was to create a system called ELIS (electronic immigration system), named in a nod to Ellis Island.
Seven years into development, the first design of the system—ELIS 1—was such a dysfunctional mess that USCIS was forced to scrap it and start again. Another four years and a total $1 billion later, the USCIS had managed to digitize just two out of the 94 forms.
In fact, the development timeline had stretched so long that the way technology was built shifted over the course of the project. In 2005, the program began with a waterfall approach—a slightly outdated but not ludicrous idea. In a waterfall approach, everything is done in a linear fashion, with requirements gathering leading into the development of a large system, followed by testing. But by 2014, when Brian Lefler, Eric Hysen, and others from the United States Digital Service joined the team, waterfall was a pneumatic tube. ELIS 1 was in large part doomed by the fact that there was no one on staff to help move the team over to a modern agile approach, in which systems are developed in a smaller, more iterative fashion. But ELIS 2 faced an even bigger problem in the agency’s assumption that digital would unquestionably be better than paper.
This article of faith turned out to be disastrously wrong.
In some cases, based purely on the task someone was looking to complete, paper was superior to digital because it was faster and more accurate. Lefler sounded a bit awestruck as he recounted watching immigration officers work their way through immense case files, searching specifically for aliases that an applicant might have used. Some applicants use multiple names for cultural reasons, so this is an issue that comes up quite a bit, but immigration officers have developed a simple system for resolving it. As Lefler explains:
They will put a little thing on their thumbs so that they can rifle through a giant pack of documents very quickly. The operator has memorized anything that is a government-looking document, that might have a name on it. As they see a document at a glance that’s probably going to have a name on it, they’ll check the name and then they’ll continue rifling through it. They go through giant stacks of paper extremely fast. There was no way to implement that electronically, short of five years of machine learning and OCR [optical character recognition] technology. It cannot be done.
A common mistake people make when trying to improve or modernize something is believing that digital will always be better. But digitizing a broken paper process doesn’t make it better. Sometimes it even makes it worse. In the case of ELIS, the team members believed their job just was to take what was on a form, digitize it, and call it done—there was no place adding sticky notes or placing the folder in a special filing spot based on where it needed to go next. They did not factor in the colossal amount of filing, categorizing, and handwritten note making that the people processing forms did on a daily basis. They didn’t think about how digitization would affect the people who used the forms every day and how. USCIS employees marked them up with notes and flagged ones that needed additional work, or put sticky notes on forms that required interviews, or placed the ones that needed a supervisor to take a look on a special shelf in the office. In digitizing only the forms, ELIS accounted for a small percentage of the work required to move immigration forms from start to finish. Without the remaining pieces, the forms simply couldn’t move.
“Digital was not better than paper because the system assumed a process that was different from how adjudicators actually did the work,” says Vivian Graubard, a founding member of USDS (and our former colleague at New America, which is a partner with Slate and Arizona State University in Future Tense). “These files were huge but also they often had to flip back and forth between pages. So it was faster for adjudicators to print out the entire application and review that way than to review on screen.”
Dana Chisnell, a user experience designer and researcher who was also part of the USDS team, describes what she found when she arrived onsite. “They’d paid no attention whatsoever to the usability of the system, and there was no vocabulary to paying attention to the needs of users.”
In other words, they tried to design the system without interviewing the front-line government workers who would be using the system. The ELIS team had thoroughly documented business processes and data flows, but none of the developers really understood how immigration officers did their jobs. The team did have subject matter experts advising them, but in many cases they hadn’t been in the field in a long time. They also rotated out every few months, so people got different information based on which SME they worked with. Along with Mollie Ruskin, another USDS designer on the project, Chisnell took a trip to a service center in Nebraska to get a better handle on what people in the field did with the paper forms. The people in the service center seemed thrilled to see Chisnell and Ruskin. No one from headquarters had ever asked how they did their work.
But after the designers returned from their trip out to the service center full of new information, HQ saw the value of visiting service centers and began regularly sending teams. Chisnell also suggested to the leadership on ELIS that it would be helpful for programmers on the project to see firsthand how immigration officers worked. The first group came back excited to implement their new findings. Once they got to work, they were able to build functionality faster and more accurately, now that they had a better understanding of what people out in the field needed to do in their jobs. The team ended up creating a schedule to get everyone out for a field visit.
But even with all the improvements made by both the USCIS team and USDS, ELIS 2 dragged along in a semi-usable state for years. By 2016, after the system had been rolled out for additional forms, immigration officers were using ELIS grudgingly and with a fair amount of seething hatred. One field office made a video of themselves kicking a computer with “ELIS” taped to it on paper.
In addition to the venting, immigration officers developed a series of work-arounds to make ELIS usable. Their offices were often filled with stacks of files in varying locations to make up for the lack of a filing system in ELIS. The system was slow and prone to outages. For a period of time in 2016 one of the most-used forms was taken offline while a team worked on stabilizing it. Basements staffed with contractors tasked with clicking a single button or unsticking cases that were caught up accidentally due to a faulty ELIS algorithm were filled to overflowing in Arlington, Virginia, costing USCIS uncounted stacks of dollars a day.
One employee at a processing center noted that a large portion of her day was occupied by undoing what ELIS had automated for her. “I spend three and a half hours every morning un-assigning the cases that don’t have evidence and going through the ones that do,” she told researchers in September 2016.
In attempting to digitize the immigration process, USCIS had taken on a massively complex analog system. Each of the 94 forms required its own process. Those forms were shipped across the country from storage spaces to field offices and service centers in order to serve millions of applicants every year. Adding to the complexity, Mark Schwartz says, USCIS wasn’t entirely sure it was going digital, beyond the vague notion that things would be better on a computer. Without hard metrics guiding their goals, the ELIS team never knew whether they’d been successful.
“They had the idea that they needed to get off paper, but they had all sorts of expectations about why—what they would accomplish by getting off of paper—and it wasn’t clear what the priorities were or how they were going to actually link getting off of paper to accomplishing those particular benefits,” Schwartz said.
The then-director of USCIS, Léon Rodriguez, struggled to keep abreast of the massive technology project running off the rails that had been deposited at his feet after his confirmation hearing in 2014. Rodriguez had worked in government for decades, but he’d never before encountered a technology project that was “going off a cliff.” He told us that “the level of problems that we kept having with ELIS was unprecedented in the face of all my prior experiences. Even in county government I had never seen the kinds of repeated problems, almost to the day that I walked out the door, that we had with ELIS.”
Rodriguez was an expert in policy and law, but technology had not been a big part of his management concerns in prior posts. ELIS was the first time he’d heard the term agile” Reflecting on what could have been done to help get his arms around the responsibility of righting a technology project gone wrong, Rodriguez says he wishes he could have had something like a technology translator to lay out significant issues in nontechnical terms—a dedicated senior person to be his ELIS liaison, in the way government officials often have policy liaisons. But there was no one at USCIS to play that role. In fact, the role doesn’t normally exist, because it is only relatively recently that technology has become the common medium for policy delivery.
Power to the Public: The Promise of Public Interest Technology
By Tara Dawson McGuinness and Hana Schank
According to Rodriguez, “You [need to] have somebody at a very senior management level who understands what’s going on with the technology development and can translate it for you. In retrospect, I would have wanted to have somebody around who was consistently available, watching what was going on with those issues.”
The gap that Rodriguez identifies is one that many public sector organizations struggle with as technology plays an ever-increasing role in how the world conducts business—how to oversee a technology project if you have never done so before and lack technical knowledge. Rodriguez’s suggestion that agencies establish a technology translator–type role would certainly solve this problem, and it is an idea we have brought before Congress. In practice, this would be an executive-level role whose sole purpose is to keep tabs on any large, mission-critical technical projects within an agency. This means that any agency running a vital project that involves a technical build should ensure that there is someone on staff with a technical background—this doesn’t need to be someone who can write code, but a person who has experience launching products and systems—who can think strategically about technology as it relates to policy and problem solving.
Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.
