I’m Sorry, Dave, I’m Afraid I Can’t Make a U-Turn

Should we be worried that our cars are controlled by software?

Also in Slate, Tom Vanderbilt explains why bad drivers are more dangerous than bum Toyotas.

Toyota Prius

When Toyota Prius owners take their cars to the dealer under the company’s latest recall, they’ll likely be out of the shop in about 30 minutes. Even though the recall has to do with a problem with the Prius’ brakes, mechanics won’t have to do much to fix the cars—they don’t have to remove the wheels, poke around under the hood, or get near the brakes at all. That’s because the flaw in the Prius is not mechanical. It’s a software glitch.

The Prius, like other hybrids, uses regenerative braking—when you hit the brake pedal, it repurposes some of the vehicle’s energy to charge up the car’s battery. In some versions of the 2010 Prius, the code that runs the braking system is buggy. It sometimes lags before applying full stopping power. To fix the error, the dealer simply downloads and installs a new version of the software—pretty much the same routine you’d go through to fix a security flaw in Microsoft Windows.

Does this sound unusual? It’s not. Modern cars are loaded with code. By some estimates, new cars contain as much software as desktop PCs, with thousands of individual functions now powered by computers. Software controls the mechanism that lets you unlock your doors, adjust your seats, and start the ignition. There’s software in the powertrain—electronic components fire the sparks, determine the correct transmission gears, and constantly adjust subtle driving characteristics to optimize fuel economy. Many safety systems—airbags, speed throttlers, electronic stability control—are powered by software as well.

And like in the Prius’ braking system, computers are now an integral part of what used to be the inviolably mechanical parts of our cars. Lots of cars—including the recalled Toyotas—now use “electronic throttle control,” which essentially means there’s no physical connection between your foot and the engine. When you hit the gas, the pedal doesn’t pull a metal cable that controls the throttle, as it did in old cars. Instead, it sends an electronic signal to the throttle to let in more fuel. More advanced steer-by-wire systems will do away with the steering shaft and column. When you turn to the right, the car will send a signal to a motor to turn the wheels; in other words, your steering wheel will function pretty much like the racing wheel for your Xbox.

As someone who loves software and is often tickled by how computers have been incorporated into every part of our lives, I’ve got a visceral reaction to the computerization of our cars: I’m terrified. The Toyota imbroglio suggests that lots of others share my fears. The company says that it’s found nothing to suggest that its electronic throttle is the cause of the sudden acceleration problem afflicting many of its cars. Instead, Toyota says, the problem involves a mechanical flaw in the pedals. But customers and safety officials aren’t sure that’s true. After all, how do we really know what’s going on in those computers?

These fears are understandable. But they’re irrational. Even though it sounds scary, integrating computers into cars brings more upsides than downsides. For one thing, cars will begin to advance at the same pace as computers. The beauty of software is that it can be updated from afar. A cell phone you buy today will keep getting better as it ages, constantly picking up new functions through downloads. Now the same will be true of cars. Through software patches, they’ll keep getting better fuel economy and better safety systems. When something goes wrong, they’ll be easier to fix. Events like the Prius brake recall will soon be unnecessary. In the future, your car will be able to patch itself when something is amiss, just like your PC grabs a security fix while you’re asleep.

For many people, that’s not a comforting thought. We don’t like the thought of computers running our cars because software is opaque. We can all understand the mechanical link between a gas pedal and an engine. When a computer intervenes, though, we have no idea what it’s doing. Software could also make it harder to know when something is wrong with your car. If you hear a strange screeching sound when you turn your wheel to the right, that could be a sign that there’s something wrong with your steering column—you’ll know that you’ve got to go to the shop to get it fixed (or try to take care of it yourself). But if there’s only a computer between you and the wheel, you’ve got to trust that the machine can self-diagnose its problems—and because you don’t have access to the computer, you can forget about trying to fix it yourself.

Software is also inevitably buggy. In very complex systems, engineers regard errors as inevitable. At Microsoft, an OS is considered ready for sale when engineers determine it has fewer than 500 known bugs. And those are just the bugs we know about. Software is hard to test, and errors are sometimes impossible to find in advance. Computers can do so many things at the same time—and so many different things in response to various inputs—that we usually have no idea what they’ll do in a given real-world situation. Toyota has hired an independent engineering firm to examine the electronic throttle in its cars. But in the same way that it’s often impossible to reproduce the combination of errors that just crashed your PC, these analysts won’t be able to go through every situation that might cause the throttle to fail on the road. That’s just the price of adding software to cars. Computers make vehicles more complex, and uncertainty is an inevitable cost of complexity. As Donald Rumsfeld might say, you get a lot more unknown unknowns.

But are we human beings much better? We fail in certain predictable, repeatable, and easily preventable ways. We drive too fast or too slowly for certain road conditions. We drive while distracted, intoxicated, or sleepy. We let people drive when they’re too young and too old. Our sensory perception is limited—we have blind spots, we can’t process simultaneous stimuli, and we can’t accurately judge changing road conditions. We’re also given to irrationality, letting our emotions take over while we’re driving a dangerous machine.

There have been lots of studies looking into the causes of traffic accidents. They all come to the same conclusion—the overwhelming reason we get in crashes is driver error. The best thing about the computerization of cars, then, is precisely the thing we find most terrifying: Software is beginning to override human control. We are the most dangerous parts of the cars we drive. The less driving people do, the fewer people will die on the roads.

Software is already starting to step in for human beings who fail on the road. Many cars now come equipped with autonomous cruise control, systems that use laser or radar to determine if your car is getting too close to other vehicles and automatically slow you down if you are. We’ve also got electronic stability control—a computer constantly monitors your wheels to determine if you’re skidding and steers you back on course if necessary.

Is it scary that we’re putting in systems that let computers override human judgment? Only if you believe that human judgment is infallible. I don’t. Sure, it’s unnerving that a computer is now running my car. But I’m sure glad it’s running yours.

Become a fan of Farhad Manjoo on Facebook.