Excerpt adapted from Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin’s Most Dangerous Hackers by Andy Greenberg, out now from Doubleday. On Nov. 6, Greenberg will discuss Sandworm at a lunchtime event in D.C. For more information and to RSVP, visit the New America website.
The malware known as NotPetya hit Ukraine on June 27, 2017, and quickly became the most devastating cyberattack in history. The virally spreading code, created by a group of Russian military intelligence hackers known as Sandworm, was intended as a climactic strike against Ukraine in the years-long cyberwar Russia had carried out against its southwestern neighbor. But within hours, the malware spread beyond Ukraine’s borders to networks around the world, hopping in seconds from computer to computer using a combination of a stolen National Security Agency hacking technique, an open-source password-grabbing tool, and the hijacked updates of a common piece of Ukrainian accounting software used by practically every company that filed taxes or had business ties in the country. NotPetya ravaged every machine it touched, saturating networks and permanently encrypting PCs and servers, destroying their data. Among its victims: Maersk, the world’s largest shipping firm, lost $300 million. FedEx lost $400 million. Drugmaker Merck would eventually tally its losses at $870 million. In total, NotPetya would be responsible for $10 billion in damage, more than any other cyberattack before or since.
But there was a less quantifiable element of the malware’s damage: its effects on hospitals, and the lives of the humans inside of them.
On the June day when NotPetya began its devastating explosion across the globe, Jacki Monson sat in a conference room in an office park in Roseville, California, a suburb of Sacramento, trying to figure out how to save dozens of hospitals from the cyberattack’s effects.
Monson served as the chief privacy and information security officer for Sutter Health, a network of 24 hospitals and clinics from Utah to Hawaii. Early that morning, she’d received a jarring message via a mailing list for the Health Care Industry Cybersecurity Task Force, a group created by the Obama administration to examine cybersecurity risks to medical organizations. The email came from the chief information security officer at the pharmaceutical firm Merck, and it warned that Merck had already been crippled by the worm—one IT administrator at the $200 billion pharma giant would later tell me that the company had lost 15,000 Windows machines to NotPetya in 90 seconds. By 9 a.m. Pacific Time, Monson was on a tense conference call with health care security executives around the world, all hoping to somehow spare their own networks from NotPetya’s wave of disruption.
Half an hour into that meeting, Monson received another call from Sutter’s head of health care information management systems. Sutter hospitals still didn’t seem to be infected with NotPetya, Monson was relieved to hear. Instead, they were facing a less obvious problem: For the last hour, the services of a company called Nuance had been down—and with them, the ability of every doctor at every Sutter hospital to dictate changes into patients’ medical records.
Nuance builds speech recognition software. The company’s code was used in the first version of the iPhone’s Siri, for instance, and the voice command system in Ford cars. By 2017, however, much of the company’s business came from hospitals, whose doctors could read changes to medical records into Nuance’s software, which was designed to automatically turn those audio clips into updates to patients’ records.
On June 27, NotPetya sprang out from Nuance’s Ukraine office to instantly paralyze the company’s digital systems across its 70 locations, from India to Korea to its headquarters in Burlington, Massachusetts. And just as at Maersk, Merck, and FedEx, desperate IT administrators would struggle for weeks to recover Nuance’s thousands of PCs and servers encrypted by the worm. “It was trench warfare,” one former Nuance staffer who participated in the rescue effort told me. “The office was in a state of triage. People were working 24/7. Every empty conference room had beds in it.”
Ultimately, Nuance would report a loss of $92 million from NotPetya, just a fraction of the damage to larger firms like Merck and FedEx. But Nuance’s transcription service for electronic medical records, aided by the company’s team of human transcriptionists, was used by hundreds of hospitals and thousands of clinics around the world. And that’s where the real toll of its outage would be felt.
Monson quickly began to see the seriousness of Nuance’s bottleneck. The service had failed silently: All across Sutter’s hospitals, doctors were reading changes into Nuance’s transcription service—in some cases, hours of audio at a time—and unbeknownst to those physicians, none of their changes would show up in patients’ files. People scheduled to go into surgery that morning might not have the final approvals they needed to be cleared for their operations. Others, like transplant recipients whose doctors constantly monitor and adjust their drugs, might miss crucial changes in treatment.
Sutter’s emergency response team soon began racing to sort through thousands of patients’ records at dozens of hospitals, trying to identify which ones might face serious consequences from their Nuance choke point. Meanwhile, Monson and her IT colleagues were desperately searching for an alternative system that would allow their hospitals’ doctors to keep making changes to health records at their normal pace. Though Nuance’s human-aided dictation services were offline, its fully automated software, installed on Sutter’s own systems, was still working. But that software was error-prone and struggled with accents. The hospitals’ own transcriptionists were overwhelmed.
It would take Sutter two weeks to switch to one of Nuance’s competitors. And within just 24 hours, Sutter was facing a backlog of 1.4 million changes to patients’ records, every one of which might have a real impact on the health of a human being.
Six thousand miles to the east, at NotPetya’s epicenter in Ukraine, hospitals were grappling with the crisis more directly. At 10 p.m., Mikhail Radutskiy, the president of a group of Kiev hospitals known as Boris Clinic, was brushing his teeth in the bathroom of his house in the western suburbs of Kiev when he got the NotPetya call from his IT administrators. He drove into the city to find that his hospitals had been hit hard: Virtually all their Windows machines were now encrypted, though medical equipment running Linux and IBM operating systems had been spared.
All upcoming appointments had to be canceled. The GPS for locating the hospitals’ ambulances was dead. The IT administrators had a full backup of their systems from three days earlier. But every test that had been performed since then, from blood analyses to MRIs to CAT scans, would have to be redone.
Radutskiy didn’t go home that night. By morning, angry patients with canceled appointments were collecting in the clinics’ lobbies, hallways, even the waiting room outside his office. “It was a mess,” Radutskiy told me simply. “It was chaos.”
But it wasn’t only Ukrainian hospitals whose computers were paralyzed by NotPetya. In some rare cases, American hospitals’ networks were fully infected, too. Heritage Valley Health System, a small two-hospital system in Pennsylvania, had itself been infected by the worm. According to one of the IT staffers at those hospitals who spoke to me, its administrators had been logged in to a Nuance server at the time of the company’s infection, allowing the worm to spread directly into the hospitals’ own systems. Before 8 a.m. Eastern Time, it had corrupted 2,000 computers and hundreds of servers.
According to that Heritage Valley staffer, equipment like X-ray machines and CT scanners weren’t running Windows, so they weren’t infected. But the shutdown of every Windows machine nonetheless crippled the hospitals’ operations. “The MRI didn’t get touched. But the computer that has the software to get the MRI image off the machine, that got hit,” he told me. “Tests are no good if you can’t see the damn things.”
Both Heritage Valley hospitals continued to serve existing patients, but new patients were turned away for around three days, the staffer said. The Associated Press reported that some of the hospitals’ surgeries had to be delayed. One woman, 56-year-old Brenda Pisarsky, told the AP that her gallbladder surgery was interrupted by a hospitalwide loudspeaker announcement calling staffers to a “command center” to deal with the NotPetya crisis.
“Europe or somewhere in that vicinity hacked into Beaver Medical Hospital and Swickley Hospital and shutdown all their computer system! It happened right after I got into the operating room!!!” Pisarsky wrote on Facebook. “Thank God no computer was used for my type of surgery. Others weren’t so lucky and had to be cancelled.”
Among American hospitals, at least, Heritage Valley’s case was an outlier. But the vast majority of U.S. hospitals that suffered from NotPetya, like Sutter Health, felt its effects not from their own malware outbreak, but from Nuance’s. One call to deal with Nuance’s swelling transcription backlog had more than 200 participants, Sutter’s Jacki Monson remembered.
In Sutter’s case, Monson claimed, the hospital network ultimately tracked down every urgent case and made sure doctors and IT staff updated medical records in time to prevent harm. “Fortunately, because of how proactive we were, we didn’t have any patient safety issues,” she said.
But not every hospital staffer was so sure. One IT systems analyst at a major American hospital—she declined to tell me which one— had a more troubling story to tell. After NotPetya’s outbreaks, she had initially focused on how to prevent her own institution from getting infected. It was only one afternoon a week later that a furious co-worker on the edge of panic had alerted her to two children’s diagnostic reports that were missing from their medical records due to the Nuance outage. Both kids were scheduled for treatments whose safety depended on their records being up-to-date. One had been transferred to another hospital for surgery the next morning.
The IT staffer felt the blood drain from her face. Did her hospital even have a copy of the dictated record changes? Would they have to delay a potentially life-saving procedure? With only hours to spare, she located the hospital’s own raw archive of the dictations, listened to close to 40 audio files, located the crucial one, and sent it out for transcription by a backup service, barely squeezing in the request in time for the child’s surgery to proceed the next day.
Over the next week, the IT staffer found two more cases where pediatric patients’ medical records were missing dictated reports, each time with only a day or two to spare before a major treatment was scheduled. In one case, a doctor had to manually retype his dictation after reexamining an ultrasound scan of a child’s heart.
In all four cases, the IT staffer told me, the hospital managed to deal with its glitches in time to prevent any delay or incorrect treatments. But even a year and a half later, she told me that those cases, where children’s care was put in jeopardy by a cyberattack, continued to haunt her.
The hospital’s Nuance outage and its effects had dragged on for more than four months. Yes, the four cases she’d seen had happy endings, she told herself. But what about the hundreds of other hospitals affected by NotPetya, and their many thousands of cases? After her own close calls, could she really believe that not one among those thousands of patients had been harmed? “I can’t say how many patients were affected or what health problems might have been caused as a result of the Nuance outage,” she told me. “But there’s a huge potential for it, just by the number of reports impacted, how long they were impacted, the critical nature of the care being provided.”
Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin’s Most Dangerous Hackers
By Andy Greenberg. Doubleday.
Slate has relationships with various online retailers. If you buy something through our links, Slate may earn an affiliate commission. We update links when possible, but note that deals can expire and all prices are subject to change. All prices were up to date at the time of publication.
If delays did occur in even a tiny fraction of those cases, the damage to human lives could have been real, argues Joshua Corman, an Atlantic Council security researcher who also served as a member of the Health Care Industry Cybersecurity Task Force. He points to a New England Journal of Medicine study that showed that even a traffic delay of less than five minutes in an ambulance caused patients to die 4 percent more often in hospitals over the following 30 days.
“Think of every hospital in the U.S. that uses Nuance. Think about how many days it was down, multiplied by the number of lab results, transfers, discharges, and how many of those are time-sensitive,” Corman said. “In some cases, time matters. Pain level is affected. Quality of life is affected. Mortality is affected.”
Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.
