Bots Till You Drop

Comparison-shopping search engines are good for consumers, bad for monopolists. 

This is the fourth in an occasional series on the economics of e-commerce. Click here for the piece on group-shopping, here for the one on eBay, and here for the one on Priceline.    

One of the most promising developments in e-commerce is the proliferation of so-called “shopping bots.” The Jetsons it’s not, but these “bots” are very good at what they do: find the lowest price on the Web for a particular item. In fact, there’s really no good reason to buy something online without using a bot to see if it’s being offered somewhere else for less. Here’s how they work.

Suppose you want to buy a Palm V. A leading shopping-bot Web site such as (Yahoo!, AOL, and most portals offer similar services) can troll the Web to provide, almost instantly, an up-to-date list of price and availability of that item at dozens of e-tailers. For instance, you’d find that the Palm V is available for $275 at, $300 at, and $375 at Aren’t you glad you asked?

Of course, as the people who bring you the Yellow Pages have pointed out—Let Your Fingers Do the Walking!—the idea of finding a good price was not unknown in the pre-Web era. But shopping bots are better because they take only two or three minutes to deliver a low price vs. the time-intensive exercise of phoning a bunch of merchants.

But shopping bots have another, greater potential in that they may force conventional retailers to offer lower prices. That is, even if consumers had all the time in the world to find the lowest price at a local retailer, they’d still be likely to find an even better price using a service such as mySimon. Here’s why.

An ordinary retailer sells just to the people who live within driving distance of his store. If he thinks up a clever idea to reduce overhead (meaning that he can offer lower prices), his reward is that people will drive a little farther to his store, and he may slightly increase his business volume. But if the manager of a Web store cooks up a clever way to reduce overhead and lower his prices, he captures the entire national market—assuming that everyone is using mySimon or something like it. In other words, services such as mySimon give businessmen a much bigger incentive to be more efficient.

Even more important, the Web’s shopping-bot services free people from the trap of patronizing the relatively small number of stores that happen to be located in their immediate neighborhood. You can think of every physical store as possessing a type of quasi-monopoly that benefits retailers and hurts consumers. That is, suppose it’s possible for a local store to sell a VCR at $150 and still turn a respectable profit. Suppose I own a similar store in the next town and price the VCR at $155. I’ll get away with the higher price, because residents of my town will probably conclude that driving a long way to save $5 isn’t worth it.

The owners of the lower-priced store will probably reach the same conclusion and price the VCR at $155 as well. That’s because if they sell at the higher price, residents of that town have no option but to pay the premium. In fact, every local store will go through the same reasoning and will charge the premium, meaning that the prevailing price at bricks-and-mortar stores will be $155. Of course, this monopoly power isn’t unlimited. Suppose I got greedy and priced the VCR at $160. In that case, a savvy businessman might decide to open a slightly cheaper electronics store in my town to steal my customers. But so long as local retailers don’t get too greedy, they can get away with charging just a few dollars more than the minimum price needed to cover overhead and get a small profit. No competitors will appear because it’s quite expensive to set up a new shop in a location where a similar shop exists—even if the existing shop is charging slightly inflated prices. (This is a slight simplification, since discount stores will spring up in the far suburbs, where land prices are lower, but the basic idea is sound: The hassle of driving means that local merchants have a limited monopoly over residents.)

The Web and shopping bots, which allow people to pick the lowest price from a national market, change the equation. On the Web, it’s not very expensive to set up a new business—at least compared with setting up enough bricks-and-mortar stores to credibly challenge a big chain such as Circuit City or Best Buy. And the payoff for setting up a Web store with rock-bottom prices, if everyone is using mySimon, is now huge. A retailer who manages to offer a lower price, even if this means a lower per-item profit, can make it up on volume. Instead of winning the business of a single town, this businessman can win the business, in theory, of the entire country. In short, a “winner-takes-all” market translates into lower prices for customers.

In fact, the comparison-shopping engines are especially good news for smaller operations lacking the deep pockets required to mount a large-scale, old-media advertising campaign. If a service such as will cough up side-by-side prices for—which buys all sorts of expensive advertising space—and, it makes sense that, with lower expenses, will win the price war. (This leads to the delightful possibility that all the mom-and-pop booksellers bankrupted by Barnes & Noble superstores can go online and bankrupt

It’s not surprising, therefore, that deep-pocketed online booksellers such as and have reason to fear bots. This points to a troublesome question. What’s to stop a big company from paying mySimon or Yahoo! to block out smaller firms? Nothing, actually. Both AOL and the Go network’s shopping bots list only merchants who pay to be included. mySimon does not require payment, but they do admit to giving preferential treatment to paid advertisers. If these shopping bots list only companies with large marketing budgets, of course, then a lot of cost savings associated with the technology won’t materialize.

Of course, it remains to be seen whether the most successful comparison-shopping search engines will continue to give preferential treatment to firms that are willing to pay. It seems possible that a Web site that relies only on ordinary advertising could make enough to pay the bills, and it would certainly have more credibility among consumers. After all, the shopping-bot sites are just specialized search engines—once you’ve developed the technology, it’s relatively cheap to operate. And it’s now received wisdom that a search engine can be successfully supported by advertising, or at least that’s what Wall Street seems to be saying when it values Yahoo! at around $73 billion.