“My goal is to be like the guy who invented Velcro,” marriage researcher John Gottman once told an interviewer. “Nobody remembers his name, but everybody uses Velcro.” Gottman’s own road to Velcro-level fame started with a 1998 article in the Journal of Marriage and the Family. He and his colleagues at the University of Washington had videotaped newlywed couples discussing a contentious topic for 15 minutes to measure precisely how they fought over it: Did they criticize? Were they defensive? Did either spouse curl his or her lip in contempt? Then, three to six years later, Gottman’s team checked on the same couples’ marital status and announced that based on the coding of the tapes, they could predict with 83 percent accuracy which ones were divorced.
Soon reporters had dubbed Gottman’s research facility the “love lab,” and his powers of prognostication had increased: In another published report, he said he could pick out future divorcees 91 percent of the time based on coding a mere five-minutes of tape. Over the next decade, Gottman’s narrow, bald head, fringed by a neat, gray beard and topped by a discreet yarmulke, began to appear everywhere—on 20/20 and The Today Show, in the New York Times Magazine and the Atlantic, and in hundreds of newspapers across the country. Malcolm Gladwell devoted most of a chapter to him in his huge best-seller Blink. In a 2007 survey asking psychotherapists to elect the 10 most influential members of their profession over the last quarter-century, Gottman was only one of four who made the cut who wasn’t deceased. “Many in the field now believe that most of what we know about marriage and divorce comes from his work,” states an article accompanying the Top 10 list.
As Gottman’s acclaim has grown, I’ve many times thought that if we were brave enough, all of us marrieds and, most importantly, would-be marrieds, would take a trip to the love lab. We’d sit facing each other, video running, pulse sensors attached to our fingertips, discuss a problem for 15 minutes or so, and come away knowing the awful or joyful truth. We’d know whether to marry or—no matter how good the relationship seemed at the present—whether to pull away and save ourselves (and any children, for God’s sake) the future heartbreak. I imagined a chain of love labs around the country, scanning couples’ marital chances as mammograms screen for breast cancer.
Then, while researching my book The Husbands and Wives Club, I looked into Gottman’s research and saw that there were reasons other than a silly attachment to romance to think twice before trusting his formula—or anyone else’s—to predict the outcome of your marriage. Gottman’s “predictions” are not exactly what most of us think of as real predictions. And the way he reports them in all likelihood makes them seem much more robust than they really are.
Undeniably, Gottman has made enormous contributions to the study of marriage. Earlier sociologists focused on mate selection: the personality characteristics of who married whom, and from where, which pairs flourished (or not). But this began to seem limited. As family therapy leader Nathan Ackerman famously put it, “two neurotics can be happily married.” To back up the idea that it was the relationship that mattered, it was necessary to step into the flow, or muddle, of couples interaction—and Gottman embraced that task wholeheartedly.
When he and a handful of other research teams began videotaping couples in conflict in the 1970s, the approach was revolutionary. Instead of just asking people how they argued or resolved disputes, researchers could see and hear them in action. A math major at MIT before he switched to psychology, Gottman developed a coding system that not only tracked the content of speech but the emotional messages that spouses send with minute changes in expressions, vocal tone, and body language. Using facial recognition systems, Gottman’s code accounts for the fact that, for instance, in “coy, playful, or flirtatious interactions,” the lips are often turned down. “It looks like the person is working hard not to smile,” he writes. Conversely, “many ‘smiles’ involve upturned corners of the mouth but are often indices of negative affect.” Such meticulous parsing allowed Gottman to coin the phrase “negative affect reciprocity,” because he saw, frame by frame, the vicious emotional circles that characterize clashing spouses.
Gottman would not have become a household name, however, without his storied powers for predicting divorce. “He’s gotten so good at thin-slicing marriages,” Malcolm Gladwell enthused in Blink, “that he says he can be at a restaurant and eavesdrop on the couple one table over and get a pretty good sense of whether they need to start thinking about hiring lawyers and dividing up custody of the children.”
So what does it mean to predict divorce? For the 1998 study, which focused on videotapes of 57 newlywed couples, I assumed that Gottman had, in the first instance, sorted them into three groups—will divorce, will be happy, will be unhappy but still married—based on the conflict-variables he believed distinguished marriages that last from those that don’t (contempt, little positive affect, elevated male heart rate, etc.). Then, at six years, he’d checked to see how right, or wrong, his predictions had been. That isn’t how it worked. He knew the marital status of his subjects at six years, and he fed that information into a computer along with the communication patterns turned up on the videos. Then he asked the computer, in effect: Create an equation that maximizes the ability of my chosen variables to distinguish among the divorced, happy, and unhappy.
The upshot? What Gottman did wasn’t really a prediction of the future but a formula built after the couples’ outcomes were already known. This isn’t to say that developing such formulas isn’t a valuable—indeed, a critical—first step in being able to make a prediction. The next step, however—one absolutely required by the scientific method—is to apply your equation to a fresh sample to see whether it actually works. That is especially necessary with small data slices (such as 57 couples), because patterns that appear important are more likely to be mere flukes. But Gottman never did that. Each paper he’s published heralding so-called predictions is based on a new equation created after the fact by a computer model.
The fundamental problem is that no matter how many equations, even quite similar ones, Gottman generates, we have no real idea of his forecasting power because of the way he reports his data. In statistics, you can’t judge the predictive oomph of anything without knowing the population prevalence of the event or condition you’re studying. Here’s a simple way to see how easy it is to fall into what they call, in the field, “base-rate neglect”: Suppose you’re told that a man named John is extremely well-educated, smokes a pipe, and wears tweed jackets with patches on the sleeve—is he more likely to be a particle physicist or a janitor? A physicist, you immediately think. But you’d likely be wrong, because janitors are common and particle physicists rare. The chances that you’d happen upon a very well-educated, tweed wearing, pipe-smoking janitor are higher than those that you’d meet a physicist who meets the same profile.
Gottman talks about his equation’s “accuracy rates,” but scientists typically don’t use such language. They report false-positive and false-negative rates and then use those figures with prevalence to ascertain the effectiveness of whatever test or method is at issue. Here’s what happens when the base rate of divorce is applied to Gottman’s 1998 data set (deep breath! math alert!): The prevalence of divorce among couples married three to six years, as Gottman’s were, is 16 percent. In other words, among any 1,000 American couples together that length of time, 160 will be divorced and 840 will still be married after six years.
Then, suppose both the false-positive rate and the false-negative rate for Gottman’s equation are 20 percent (which is only an assumption, because, remember, Gottman doesn’t provide those figures; I chose it based on his assertion of 80 percent “accuracy”). False positives are couples whom the formula classifies as divorced who really aren’t, so with a 20 percent false-positive rate, Gottman would call 168 * of the still-intact couples divorced (840 x 0.20). False negatives are couples who are divorced but whom the formula misses, so with a 20 percent false-negative rate, Gottman would put 32 couples in the married column who don’t belong there (160 x 0.20). In sum, Gottman would peg 296 couples as divorced—168 + (160-32), but only 128 of those actually would be, meaning his predictions would be right 43 percent, or less than half, of the time. Much less impressive.
To bring it back to men, women, and rupture, Richard Heyman, a psychology research professor at the State University of New York who wrote with a critique of Gottman’s divorce prophesizing, uses an analogy from the stock market to explain why predicting the future of any individual couple is so tough. “It’s not like Gottman is just trying to take particular variables and predict what’s going to happen to the S&P 500,” Heyman said in a phone interview. “He is using variables to say he can predict the outcome of each individual stock, with over 90 percent accuracy.” It’s seductive to think that there are a few discrete factors—namely, how you communicate during a fight—that are going to “trump everything else in your life that might influence whether you’ll get divorced or not,” he continued. That way, you’d know what to concentrate on—if I can just do this, we’ll be fine. (Conversely, I’ll never be able to do that, so we’re finished. The predictive nuggets can be pernicious, too, squashing any hope for, or effort toward, betterment.)
There are, in fact, some spheres in which you can tell a lot by knowing just a little—for example, SAT scores and high-school grades have been shown to predict success in college far better than more subtle analyses that take into account such things as teacher recommendations. But based on the evidence we have so far, marital relations haven’t yet succumbed to such delightfully efficient approaches.
Become a fan of DoubleX on Facebook. Follow us on Twitter.
Correction, March 8, 2010: Due to a typo, this article originally misreported the final result of a formula as 1680 instead of 168. (Return to the corrected sentence.)