The Scourge of Garbage-In, Garbage-Out Research on Innovation

The embarrassing heart of economics as a discipline is that absolutely nobody has any idea what the sources of long-term prosperity are. Instead, you get phlogiston economics grounded in a mathematical model that tells us the most important thing is the unexplained residual. Then economists fight over how to label the residual. The residual is often labeled “technology” which people associate with the idea of “innovation” so it’s often deemed important to try to research where innovation comes from.

The problem is that in order to study this empirically you need some non-circular measurement of innovation, which isn’t possible in terms of the growth model. So what researchers have cottoned on to lately is the use of patents as an empirical index of innovation. The virtues of this are clear: It’s non-circular and it’s easy to look up the data. With the data in hand, you can conduct empirical studies and test models. With these empirical studies and tests of models, you can submit papers to journals and get them published. So using patent-counts as a measure of innovation solves basically all the problems an economist might have except for if you actually want to answer the question since you’d have to be certifiably insane to think that raw patent counts as a good measurement of innovation and it only gets crazier if by “innovation” you really mean total factor productivity.

As an example, my favorite place to get lunch by my office is Chopt a small NYC/DC salad chain that’s engaged in a lot of interesting business model innovation. One thing they do is that at peak demand times when the store gets crowded and things get hectic, they employ a guy to serve as a kind of “traffic cop” who directs customers to the optimal line to order at.

This might be a good idea or might be a bad one. But if it’s working for them, it’s a real example of TFP-boosting innovation. It does, yes, involve adding a little bit of extra labor (the traffic cop) to the production function. But it’s not highly skilled labor. And the point of it isn’t a linear addition where you have one extra guy making salads. Instead the idea is that at peak demand times, employing a traffic cop lets everyone work more efficiently. The same stock of labor (cashiers, salad-makers) and capital (physical space, mezzalunas, cash registers) is producing more salads by having the customers flow more efficiently through the space. And yet this innovation is not patentable and it would be a public policy disaster to grant a patent for it.

I was thinking of this recently because of the newish paper from Daron Acemoglu, James Robinson, and Thierry Verdier which argues that it would be bad for the world if more rich countries had a Nordic social model. Obviously lots of people with left-of-center political views don’t like the Acemoglu/Robinson/Verdier conclusion and so are eager to find objections to their analysis. One thing many people have seized on is that at a key stage in their argument they rely on a patent-count as an index of innovation, and so I’ve seen several people note that this is ridiculous. And it is ridiculous (for starters two of Sweden’s biggest firms, Ikea and H&M, operate in the design sector and are ineligible for intellectual property protections). But the problem is that it’s not just ridiculous, it’s a standard procedure in the field. The idea is basically that measuring innovation is so important that it’s better to use nonsense data in your models than to just admit that you can’t do it. But that’s backward. Precisely because innovation is so important, it’s incredibly damaging to build powerful conclusions on a foundation of nonsense. It is genuinely problematic for people who’d like to publish articles on the subject that there is no useful empirical measurement of innovation available, but that’s tough on you.

What we’re getting right now is a lot of garbage-in / garbage-out analysis.