Facing Up to Our Ignorance

Diane Ravitch shares a timely lesson: We don’t (yet) know how to save schools.

Barack Obama inherited George W. Bush’s war—not just in Iraq and Afghanistan, but also in the nation’s public schools. Bush’s battle plan was to criticize teachers’ unions, increase the number of charter schools, and promote testing and accountability through No Child Left Behind, his signature legislation. Over the past year, Obama has done more than continue to prosecute his predecessor’s war. In a surprise to many, he has quietly escalated it, even as the intelligence on which this war was founded has come under increasing scrutiny. If few have noticed, well, the country has been a little distracted by health care reform and the disastrous economy. But as Diane Ravitch argues in The Death and Life of the Great American School System, it’s time we paid attention. Much has already been made of Ravitch’s book, in which she has renounced her long-standing support of charter schools and NCLB. Her change of heart is notable because she is one of the nation’s most serious and credible education scholars. In a career spanning four decades, she’s written multiple histories and been an influential voice in policy debates, challenging both the reflexive right and left at different points over the years. But like a general who can no longer ignore the mounting casualties of a war she helped to propagate, she now argues the battle cannot be won under the current terms of engagement. In this sense, her book arrives with the force of the Pentagon Papers. Ravitch, of course, isn’t revealing state secrets—but it sometimes seems as if she is, given how counter much of the evidence she presents runs to prevailing wisdom about education reform.

Ravitch takes aim at the centerpiece of the Obama administration’s education surge: a crusading confidence that data—derived from testing—can tell us all we need to know not just about what’s wrong with failing schools, but how to fix them. The view underpins Obama’s primary offensive, his “Race to the Top” fund. States submit proposals, and the winners get a chance to take home some of the unprecedented $4 billion in federal stimulus money earmarked for reform. But in order for states to be considered, they must not only lift caps on the number of charter schools in their states but also start collecting more data than required under NCLB, to guide a last innovation: performance pay for teachers, which states must likewise agree to embrace. Congress and most state legislatures have historically opposed unlimited charter schools and merit pay, but in these cash-strapped times, few states can afford to say no.

In addition, this past week Obama announced an initiative that would direct nearly $1 billion of federal aid to 5,000 failing high schools, provided their districts take drastic measures, such as firing half of a school’s teachers or immediately turning the schools into charters. Ravitch tweeted in response: “If we close 5,000 low performing schools, where will we find 5,000 great new principals and hundreds of thousands of great teachers?”

She argues that if we think the data hold the answer, we’re indulging in wishful thinking, and her case should be enough to unsettle the best and the brightest as they take their turn at reform. Teacher quality, recruitment, and retention have become the new frontlines of this reform, spurred in part by powerful computer programs that can now mine mountains of data with relative ease. Ravitch focuses on the analyses of William Sanders, then a statistician at the University of Tennessee, whose previous work was in agriculture, manufacturing, and engineering. Drawing on past studies rather than on visits to classrooms, he decided that the single most important variable affecting students’ performance is their teacher. (This is contrary to much research that suggests levels of poverty, health, parental education, and peer influences have equal or greater impact.) Since teachers’ abilities vary widely, Sanders developed models for performing what is known as “value-added assessments”—that is, determining how much specific teachers boost achievement (based on test scores) for specific students over time.

Enter Stanford economists Eric Hanushek and Steven Rivkin, whose further explorations Ravitch scrutinizes. * They looked at how different variables—certification, general education level, salary—affected teacher quality. Nothing, in their judgment, was predictive. “A good teacher,” they concluded, “would be one who consistently obtained high learning growth from students, while a poor teacher would be one who consistently produced low learning growth.” So schools, they argued, should just open the profession to anyone and see who sinks or swim. They offered a tidy formula for how swimmers could be saviors: If a student had a teacher in the 85th percentile of teachers for five years in a row, they calculated, this would be enough to eliminate the persistent “achievement gap” between low-income and high-income students—the still-elusive grail of reform.

Hanuchek and Rivkin understandably want to improve the quality of instruction in impoverished schools, which undeniably can become repositories for the worst teachers. What Ravitch doubts is that this intervention all by itself can realistically promise to turn around failing schools in such extraordinary fashion, without any attention to other variables that affect student outcomes. And as a practical matter, she asks, how are schools—especially in inner-city neighborhoods—supposed to attract these large stables of consistent superstars? As Ravitch writes, “This is akin to saying baseball teams should consist only of players who hit over .300 and pitchers who win at least 20 games every season; after all, such players exist, so why should not such teams exist.” The model also assumes yearly gains are cumulative, when most studies show that students (particularly low-income students) backslide substantially during summer months. At the very least, such an intervention would likely need to be coupled with efforts to extend the school day and school year.

But perhaps most damning, Ravitch writes:

No school or school district or state anywhere in the nation had ever proved the theory correct. Nowhere was there a real-life demonstration in which a district had identified a top quintile of teachers, assigned low-performing students to their classes, and improved the test-scores of low-performing students so dramatically in three, four or five years that the black-white test score gap closed.

Yet suddenly this one theory is driving reform. This is why Obama wants states to gather more data. He wants to tie individual students’ scores to individual teachers, so achievement can be monitored not only school by school but also classroom by classroom, and then teachers can be fired or given raises accordingly.

Ravitch, however, argues that the past eight years should leave us wary of relying solely on isolated test results for high-stakes decision-making. She isn’t against tests per se—or even tests being used as one element in a more comprehensive evaluation system. Schools have to have some way of judging performance, and Ravitch praises the National Assessment of Educational Progress, or NAEP, given periodically to representative samples of students, as the gold standard. But the state tests developed under NCLB, she maintains, are terrible. They are devoid of real content and have created a culture of test-prep that is inimical to real education. Such arguments are not new—indeed, many of Ravitch’s old opponents (back when she was on the pro-NCLB side) have been making this case for years.

What is new is the thoroughness with which Ravitch debunks heralded gains of student achievement under NCLB. For example, in New York City, where Mayor Michael Bloomberg and schools chancellor Joel Klein have trumpeted substantial rises in student scores, Ravitch reveals the state simply lowered the score required to be judged “proficient” in reading and math. (She notes that scores in New York City went up even as scores on the NAEP began to stagnate.) And in Arne Duncan’s Chicago, where he was superintendent of schools before becoming Obama’s secretary of education, big jumps in students’ scores again proved illusory when compared with the NAEP. Nor did they persist into high school. This has been the pattern throughout the country. In 2007, Texas reported that 85.1 percent of its fourth-graders were proficient in reading, but the NAEP found that only 28.6 were. Tennessee reported 90 percent of its students were proficient, but the NAEP found that only 26.2 percent were. And so on.

The data, as Ravitch says, disappoints on other fronts, too—not least in failing to confirm high hopes for charter schools, whose freedom from union rules was supposed to make them success stories. To the shock of many (including Ravitch), they haven’t been. And this isn’t just according to researchers sympathetic to labor. A 2003 national study by the Department of Education (under George W. Bush) found that charter schools performed, on average, no better than traditional public schools. (The study was initially suppressed because it hadn’t reached the desired conclusions.) Another study by two Stanford economists, the Walton Family and the Michael and Susan Dell Foundation (staunch charter supporters), involved an enormous sample, 70 percent of all charter students. * It found that an astonishing 83 percent of charter schools were either no better or actually worse than traditional public schools serving similar populations. Indeed, the authors concluded that bad charter schools outnumber good ones by a ratio of roughly 2 to 1.

Obviously, some high-visibility success stories exist, such as the chain run by the Knowledge Is Power Program, or KIPP, which I’ve previously discussed here. But these are the decided exceptions, not the rule. And there’s no evidence that a majority of eligible families are taking advantage of charters, good or bad. “While advocates of choice”—again, Ravitch included—”were certain that most families wanted only the chance to escape their neighborhood school, the first five years of NCLB demonstrated the opposite,” she writes. In California, for example, less than 1 percent of students in failing schools actually sought a transfer. In Colorado, less than 2 percent did. If all this seems a little counterintuitive, Ravitch would be the first to agree. That’s why she supported charters in the first place. But the evidence in their favor, she insists, simply hasn’t materialized.

Ravitch isn’t arguing for pessimism. She’s arguing for humility—the kind she’s had to learn the hard way—and a healthy skepticism about silver-bullet notions of reform. The dirty dark secret of NCLB is that we may know how to identify the worst performing schools, but no one (yet) knows how to turn them around in any consistent and reliable way. And I mean no one. Not the Gates Foundation to date. Not most charter programs. No one.

As one study Ravitch cites concludes: “The only guaranteed strategy [for improving schools] is to change the student population, replacing low-performing students with higher-performing students.” And this is, in fact, what the rare success stories—like KIPP—typically do: skim off the best and most motivated students from disadvantaged neighborhoods. These best students deserve better options, but this approach doesn’t address the larger problem of how to fix chronically failing schools.

Ravitch’s realism should be a bracing force in the dogged reform efforts—not crusade—we will be shouldering for a long time to come. Still, Obama, and other new voices in the debate, are surely right that teacher quality and recruitment are important ingredients of comprehensive reform. Near the book’s end, she conjures up the ghost of her own beloved high school English teacher, Ms. Ratcliff, and Ravitch might have gone on to reflect on the main difference between then and now. In the 1950s, smart women, except for truly determined trailblazers, had few professional options beyond teaching. Ditto for blacks and other minorities. If you had a particularly smart and ambitious daughter, people would say, “I bet she grows up to be a teacher!” While many things have happened to public schools over the last 50 years, one of the most important is that this low-cost captive labor pool of extremely talented men and women has evaporated completely—and along with it the respect that was once automatically accorded to those who entered the profession. Today, with so many more (and better-paying) careers to choose from, it’s unclear Ms. Ratcliff would be a teacher at all.

Ravitch is absolutely right to caution that trying to evaluate teachers solely on the basis of crude standardized tests is too reductive and will likely only alienate the kind of creative, dynamic people the profession hopes to attract in greater numbers. But the question remains: How do we lure more, talented people to the profession and give them—and the many superb teachers who already exist—the support and respect they deserve? For all its faults, if NCLB has done nothing else, it’s helped to clarify this challenge.

Correction, March 19, 2010: This article originally misspelled Eric Hanushek’s name. Return to the corrected sentence. (Return  to the corrected sentence.)

Correction, March 26, 2010: This article originally stated that the Eli and Edythe Broad foundation helped fund a study by two Stanford economists. Actually, it was the Michael and Susan Dell Foundation. Return to the corrected sentence.

Become a fan of Slate on Facebook. Follow us on Twitter.