The Magic Question

Judges trust jurors to assess their own biases. That might not be such a good idea.

The prosecution team at their table during the proceedings where District Court Judge William Sylvester entered a Not Guilty plea on behalf of Aurora theater shooting suspect James Holmes
Aurora theater shooting suspect James Holmes (center) on March 12, 2013.

Photo By RJ Sangosti/The Denver Post

This week, hundreds of Americans are skipping work in order to audition for starring roles in three high-profile criminal trials being staged around the country. The defendants in all three cases—the alleged killer of 6-year-old Etan Patz, in New York; the alleged Aurora shooter, in Colorado; and the alleged Boston Marathon bomber, in Massachusetts—stand accused of committing crimes that have attracted national attention, making the task of finding people who can evaluate the evidence against them in an unbiased way extraordinarily difficult. But even in more run-of-the-mill cases, in which the contaminating potential of pre-trial publicity is not an issue, seating a panel of 12 open-minded jurors is more challenging than you might think. And according to some critics, the process our criminal justice system relies on to find those jurors is seriously flawed.    

People inevitably bring biases with them when they’re summoned to the courthouse for jury duty. Judges try to overcome this obstacle systematically, by asking each potential juror a series of pointed questions about who they are, who they know, what they’ve experienced in life, and how they think about crime and punishment. With attorneys for both the prosecution and the defense keeping close watch—and in some jurisdictions, asking their own questions—judges listen for any indication that a prospective juror might be incapable of rendering a fair verdict. When they identify someone who does seem compromised for one reason or another, they dismiss that person “for cause,” and move on to whoever’s next in line.  

The problem, critics say, is that most of the questions that get asked during the so-called voir dire process—whether the would-be juror has ever been the victim of the crime that the defendant is being charged with, for instance—usually end up taking a backseat to one query in particular, which caries outsized power. It’s known in some circles as the “magic question”: Do you believe you could be fair and impartial in administering a verdict in this trial?

It seems like a perfectly reasonable thing to ask. But what worries some experts is that, all too often, judges are inclined to automatically accept everyone’s answer as truth. When interviewing prospective jurors whose attitudes and life experiences raise red flags, but who insist that they can be fair anyway, research shows that judges often choose to defer to the juror’s self-assessment. This is troubling, critics argue, because answering the “magic question” accurately requires a level of self-knowledge and introspective clarity that most of us just don’t have.

“Judges [act as though] the juror can sort of look into her own soul, and see whether she can be fair or not,” says Christopher Robertson, an associate professor at the University of Arizona College of Law. “But all the evidence from the field of psychology suggests that a lot of what the mind does is subconscious. … It has to be, for us to make the millions of decisions we have to make in a given day.”

Nevertheless, experiments and surveys conducted by experts on jury selection have shown that the answers people give to the “magic question” tend to have a disproportionate impact on their chances for “surviving” the voir dire process. According to a 2004 study from the National Center for State Courts, a nonprofit think tank that specializes in judicial administration, jurors in California who told a judge that they could be impartial were 71 percent less likely to be removed from the jury pool than jurors who didn’t. The authors of the report concluded that “the juror’s self-assessment about fairness is the strongest factor in judicial decision-making,” and that “judges place fairly heavy reliance on juror candor during voir dire and tend to take juror statements about fairness at face value.”

A separate study, published in 2008, reported that a prospective juror’s apparent disposition when answering the “magic question” makes a difference, too. Under the supervision of Mary Rose of the University of Texas–Austin and Shari Diamond of Northwestern University School of Law, a panel of trial court judges were asked to read a series of vignettes about prospective jurors who had asserted, with varying degrees of confidence, that they could be fair in a trial despite having certain personal conflicts related to the case. The judges in the study were then asked to determine, based on their experience of the justice system, whether the prospective jurors were likely to be dismissed or allowed to serve. What Diamond and Rose found was that confidence mattered: judges had more faith in prospective jurors who were firm and unequivocal in asserting that they could do a good job than in the ones who expressed the slightest hint of hesitation.

But is self-confidence necessarily a mark of credibility? “It’s possible the person who says, ‘Yes sir, your honor, I’m completely confident,’ is just not being very thoughtful in responding to the question,” says Diamond. “It’s a very natural tendency, to use that kind of confidence as a cue, but the problem is we really have no idea whether it’s a good cue.”

For Robertson, that uncertainty is the heart of the matter. Given the mountain of Nobel Prize–winning psychology research showing that humans are terrible at evaluating their own biases, he says, it’s important for judges to be skeptical when prospective jurors are testifying to their own ability to be fair. A juror who thinks of himself as a rational, objective thinker might harbor a subconscious inclination to convict suspected drug dealers because he has a brother who is addicted to heroin; a juror who self-identifies as unassailably egalitarian may bring to trial an unacknowledged bias toward an Asian or Hispanic or black defendant.

To determine whether a person who believes he is unbiased is actually likely to be, Robertson worked with a colleague to set up a randomized experiment in which two groups of people were asked to sit through a fake trial and decide at the end whether the defendant was guilty. Before the proceedings got underway, half the test subjects were asked to read a stack of inflammatory articles meant to prejudice them against the defendant; the other half were not. Then, before the “trial” began, all the study participants were asked to write down whether they felt capable of approaching the case with an open mind. Afterward, both the treatment group, which had seen the inflammatory articles, and the control group, which hadn’t, were asked to render a verdict.

“If the magic question works, the people in the treatment group who said they could be fair should have convicted at the same rate as the control group did,” Robertson says. “That would have proved that the filter works—that the ‘magic question’ really is magic. But we found the exact opposite.” The people who read the biasing material and still said they could be fair were just as likely to convict as the ones who said they couldn’t—their confidence in their impartiality, it turned out, was misplaced.

Robertson says the point of his findings is not to suggest that judges are cutting corners in their approach to jury selection—it’s that they’re putting too much stock in people’s self-assessments. “I think the judges really believe it works, that the answer to the magic question really is predictive of the truth,” he said. “I just don’t think they’ve assimilated what psychologists now know about implicit bias. And so, it’s not completely a cynical process. To some degree, it’s a naive process.”  

It’s worth noting that judges aren’t the only ones with power to shape the makeup of a jury. Attorneys have a role to play in the process as well, but it tends to be relatively limited: though they can veto some prospective jurors, no-questions-asked, by filing so-called peremptory challenges, most jurisdictions only allow them to play that card a handful of times. And while attorneys can try to convince the judge at any point during the process that a prospective juror is compromised in some way, only the judge can decide to actually dismiss them.

So what can be done? If we’d like to see fewer jurors with disqualifying biases “survive” the voir dire process, how can the jury selection process be changed?

According to Greg Hurley of the National Center for State Courts, some judges have started incorporating robust explanations of implicit bias into their jury instructions, explaining to jurors why it’s important for them to be honest with themselves about their subconscious prejudices, and to try to keep them in check. The hope, Hurley said, is that even if some of the 12 jurors do have prejudices that interfere with their ability to look clearly at the case they’re deciding, talking to each other during deliberations and discussing their biases openly could help neutralize some of their distorting effects.

Robertson, for his part, would like judges to simply stop asking the “magic question,” and rely instead on questionnaires designed to reveal a prospective juror’s leanings more organically. One option, he says, would be send surveys designed to reveal biases to people’s homes, and based on their responses, assign them to cases in which those biases are least likely to affect their work as a juror. A grudge against drug offenders, after all, wouldn’t prevent a person from being a productive member of a jury in a tax fraud trial.

“It’s not about good jurors and bad jurors,” Robertson says. “It’s about, well, which juror should sit on this case, and which one should sit on the medical malpractice case that’s being tried down the hall? It’s about matching more than sorting.”

On the first day of jury selection at the Pedro Hernandez trial last week, Justice Maxwell Wiley of the State Supreme Court in Manhattan made it clear he understood that many of the people gathered in his courtroom had been hearing about Etan Patz ever since his disappearance in 1979.

“For those of you who were born less than 35 years ago and know nothing about this case … try to keep it that way,” Justice Wiley told the youngsters in the crowd. Everybody else would have to just do their best: As long as they promised to put aside everything they thought they knew, they could still fulfill their civic duty. “It’s something fair-minded people can do,” Justice Wiley said.

Looking around the room, I wondered: Were these fair-minded people, or not? Surely, some of them were. But did they know who they were?