Suppose that you are a football coach and you are having an interview with an athletic director. “Vince,” says the AD—in our fantasy your name is Vince, because you are a football coach—“this is a great opportunity for a young coach. The Mountain West Conference is an up-and-coming league with a bright future. We’re offering to increase your salary almost a full 10 percent over what you’re making as an assistant in the SEC, and we’ve got a four-year-old Honda you can drive anytime you need it. We’ve just got a couple of special requirements for you.”
“Sure,” you say optimistically. “Let me hear it.”
“First,” says Gomer—in our fantasy the AD’s name is Gomer—”you can’t recruit anyone more than 50 miles from our campus. We’re trying to build up local support, see. If you bring in those local boys, that will help us build our fan base. And second, no passes over 10 yards.”
“No passes longer than 10 yards. We’re a ground game, see; always have been. When we went 7-1 back in ‘74, we only threw 19 passes all year. That’s our tradition, and that’s the right way to play football, just punch the ball up the middle. That’s the way we want to play it.”
If you have an ounce of sense or a smidgen of self-respect, Vince, you’re out the door before Gomer can get the cigar out of his mouth and start to explain that the four-year-old Honda is really a six-year-old Hyundai with a real estate ad painted on the door.
A couple of years ago, Dr. Hal S. Stern, the head of the Department of Statistics at the University of California-Irvine, wrote an article in the Journal of Quantitative Analysis in Sports “advocating a boycott of the Bowl Championship Series by all quantitative analysts.” Stern argued that statisticians shouldn’t want anything to do with the “computer rankings” that are an element of the BCS system. I am writing here to sign onto Stern’s call for a boycott. Tell Gomer to stuff it, in other words.
“It is,” wrote Stern, “generally a bad idea for quantitative analysts to remove themselves from the decision-making process,” but there are also times that, to preserve your self-respect, you pick up the briefcase and walk out of the room. This, in my view, is one of those times. The problems with the BCS are:
- That there is a profound lack of conceptual clarity about the goals of the method;
- That there is no genuine interest here in using statistical analysis to figure out how the teams compare with one another. The real purpose is to create some gobbledygook math to endorse the coaches’ and sportswriters’ vote;
- That the ground rules of the calculations are irrational and prevent the statisticians from making any meaningful contribution; and
- That the existence of this system has the purpose of justifying a few rich conferences in hijacking the search for a national title, avoiding a postseason tournament that would be preferred by the overwhelming majority of fans.
In truth, my objections to the system are a little different than Stern’s. His biggest objection, I think, is No. 4 above—that the BCS system is used to justify something that should not be justified. To me, the deal-breaker is No. 3—the imposition on the computer rankings of irrational rules that essentially guarantee the failure of the process.
But let’s take them one at a time:
1. There is a profound lack of conceptual clarity about the goals of the method.
This is reflected in the fact that the rankings are routinely described as “computer” rankings. Computers, like automobiles and airplanes, do only what people tell them to do. If you’re driving to Cleveland and you get lost and wind up in Youngstown, you don’t blame your car. If you’re doing a ranking system and you wind up with Murray State in western Kentucky as the national football champion, you don’t blame the computer.
There are several things that a ranking system could do. It could rank teams based on their accomplishments over the course of the season—whom they played and whom they beat—or it could rank them based on the probability that they would win against a given opponent. It could rank teams based on how they have played over the course of the season, including perhaps in some early-season games against teams that were not quite sure who their quarterback was, or it could rank them based on how strong they are at the end of the season. It could rank the teams based on consistency, or it could rank them based on dominance.
Which of these is the goal of the BCS system?
Nobody has any idea. It’s never been debated. There is a perception among the people who are in charge of this monkey that if you just turn the rankings over to a computer, the computer will figure those things out. The reality is that it can’t. It is very difficult to objectively measure anything if you don’t know what it is you are measuring.
2. There is no genuine interest here in using statistical analysis to figure out how the teams compare with one another. The real purpose is to create some gobbledygook math to endorse the coaches’ and sportswriters’ vote.
Throughout the 11 years of the BCS, whenever the “computer” rankings have diverged markedly from the polls, the consensus reaction has been, we have to do something about those computers. And they have; whenever the computer rankings don’t jibe with the “human polls,” they fix the computers. In 2000, the computers didn’t pick Miami as one of the top two teams. The coaches and sportswriters thought Miami should have been there, so they changed the computer system.
In 2001, according to Stern, “the BCS selected once-beaten Nebraska over once-beaten Oregon despite the fact that Nebraska had lost badly in their last regular season game. Popular perception this time was that the computer ratings paid too much attention to the large margin of victory in Nebraska’s early season triumphs while not putting enough value on Oregon’s steady but unspectacular performances.” What did they do? Fix the computers. In 2003, the computer rankings once more disagreed with the coaches’ and the fans’ and the writers’ perceptions, and so, once more, the computer rankings were fixed to prevent a recurrence of whatever the problem was.
3. The ground rules of the calculations are irrational and prevent the statisticians from making any meaningful contribution.
One of the polls used in the BCS system is the Peter Wolfe rankings—no Prokofiev jokes, please. According to Wolfe’s Web site, “A significant but hard-to-measure factor in comparing teams is sportsmanship. Running up the score is generally looked on as evidence of bad sportsmanship, behavior which should not be encouraged or rewarded. With this in mind, the BCS has chosen computer systems that use only won/loss data (and not scoring margin) to compute ratings. We have developed such a system that provides reasonable results.”
I don’t question that Wolfe is a good man doing the best he can within the BCS strictures, but this is childish pablum. The prohibition against using point differentials to rank teams, of course, dates from the Nebraska-in-2001 experience, when those dirty Cornhuskers beat Troy State, Rice, Missouri, Iowa State, Baylor, and Kansas all by 28 points or more. The BCS reacted to this by requiring the computer rankings to treat a 56-7 victory the same as a 20-17 contest.
This is very much like a situation in which a surgeon leaves a scalpel in a patient, and the hospital reacts by prohibiting surgeons from using scalpels. I understand that the point of the game is to win, not to score as many points as possible, and I certainly can understand football coaches saying, “We want a system that emphasizes winning and diminishes the importance of the score.” That’s reasonable.
But saying, “We’re not going to pay any attention to the score of the game, and, by the way, you can’t pay any attention to whether it is a home game or a road game, either”—that’s just stupid, Gomer. For football coaches to impose a rule like that on the statistical analysts is very much like the AD, frustrated by seeing long passes intercepted, telling the football coach he can’t throw passes longer than 10 yards.
Look, guys, none of us are claiming that the statistical analysts understand the game of football as well as the football coaches do, or that our analysis should take precedence over the informed opinions of experts. I’m not saying that at all.
But at the same time, statistical analysts are professional people. Heck, some of us are almost as smart as football coaches—high-school football coaches, anyway. There is no point in our participating in the process if you’re going to tell us how to do the analysis based on your ignorant, backward-looking prejudices. Run your own damned computers.
4. The existence of this system has the purpose of justifying a few rich conferences in hijacking the search for a national champion, avoiding a postseason tournament that would be preferred by the overwhelming majority of fans.
In the way that I phrased that, it sounds like the big football schools are to blame. That’s actually not the way I see it.
There is no reason why the big football schools, whose alumni fund the big programs that compete for the national championship, should have to donate the proceeds of their competition to smaller schools and to schools that choose not to fund competitive football teams. The BCS is a result not of the greed of the big schools but the self-righteous avarice of the smaller and less-committed schools that form the bulk of the NCAA.
It is inherent in the nature of sports to seek a clear resolution of the competition. You have two football conferences, two basketball conferences, two baseball leagues—you want to know who the best team really is. That doesn’t come from anywhere; it’s integral to the sport. It’s like a movie; either the boy gets the girl, or he doesn’t. Either the cop catches the killer, or he doesn’t. Either the hero wins the battle, or he dies on the battlefield. That’s just the way it is, whether it’s Shakespeare or schlock. Leaving the situation unresolved is unpopular because it’s unnatural.
In the 1990s there was a strong movement, within the NCAA, to organize a national postseason football tournament. The problem was, had the NCAA in fact organized such a championship, two other events would almost certainly have followed:
- The smaller schools, which outnumber the big football powerhouses about 5-to-1, would have voted to send a lot of the money to the smaller schools that in fact had not participated in the national championship contest in any meaningful way.
- The big football schools would have bolted and revolted. They’d have walked out of the NCAA and formed their own organization. The two-tiered system of NCAA and NAIA schools would have been replaced by a three-tiered system with the NCAA occupying the middle tier.
The creation of the BCS system was simply a less dramatic revolt. And, as I said, the BCS schools were right: There is no reason why schools that don’t fund programs to participate in the battle for the national championship should share in the proceeds of the contest.
There are two ways to get around this problem. First, the NCAA could pass a unanimous or nearly unanimous resolution, promising not to try to steal the proceeds of a national title contest and give the money to small schools, deserving nephews, or the church poor box. The BCS could then dissolve and be replaced by an NCAA Football Tournament involving eight to 16 teams, and the big football schools would wind up with just as much money or a little more.
Or, if that doesn’t work, we can pass a law creating a new National Collegiate Sports Collaborative and requiring all schools receiving federal funding to join and participate. And if we have to do that, we’ll decide how to split the money.
You don’t actually have to pass the law; just bring it up in the House of Representatives and get 150 votes, and the BCS schools and the NCAA will work out their differences faster than Christiane Amanpour can get to a war zone.
But until that happens, statisticians, quantitative analysts, and all related professionals should have the dignity, the self-respect, and the common sense to have nothing to do with the BCS. This isn’t a national championship—it’s a big-money waltz. The only role that the computer rankings play in this is that they’re there to take the fall when the system doesn’t work—and it doesn’t work most of the time. When it doesn’t, you can put the blame on the greedy small schools that wanted to milk money from the big football factories, on the greedy big schools that wanted to keep as much money as possible in the fewest possible hands, on the lunk-head football coaches who can’t program a computer to play tic-tac-toe but want to make all the rules, or on the Congress that sits idly by and watches it happen. You guys want to make a mess of this, you can make a mess of it without our help.