The mainstream media love lists. On almost any day, open up the paper, or log on to your favorite news website, and there's some sort of list: The Richest Americans, The Best Colleges, The Most Powerful Political Leaders. . . .

These lists are designed to look like valid studies. They may even tout a rigorous methodology. But most of these lists are generated using arbitrary---and often ideologically driven---criteria. The lists that I find the both most frustrating are those that rate cities: The Happiest Cities, The Best Cities to Raise a Family, and one from last month, "The Best U.S. Cities For Business," compiled by MarketWatch, which is part of the Wall Street Digital Network.

This list caught my attention in part because Des Moines, Iowa, ranks No. 1. Nothing against Des Moines (my sister and her family live there, after all), but what set of criteria would allow Des Moines to be No. 1, and put Washington, D.C.---a very different kind of city---at No. 2? The list is especially baffling when you consider that Des Moines scored 50 points (on a 1,000-point scale) more than D.C., even though less than five points separate most of the other 99 cities on the list.

The reasons for the strange rankings: bias and ideology. To begin with, it turns out that the rankings take into account 10 criteria. Each criterion is worth 100 points. With so many criteria, the cities that did best---such as Des Moines---are those that were not necessarily excellent in one or more categories, but didn't bomb any of them either. In other words, the rankings rewarded not true excellence but consistent above-average-ness. MarketWatch's Russ Britt put it this way:

"How did this small metropolis plunked down in the middle of corn country win out? Nothing flashy, really. It just had above-average scores in every one of 10 metrics that MarketWatch examined. Well, not just above average; it was in the top fourth of all but one metric."

What should we call this bias? An addiction to mediocrity? More charitably, we might call it "The Lake Woebegone Effect," where innovation and excellence are frightening, where the best thing you can say about someone is that they are "above average," but not too much above average.

Now consistent above-average-ness wouldn't be so bad if what was evaluated actually mattered. But that's the next problem with this pseudo-study. For example, one would think that the ability to produce jobs would be a solid indicator of a city's business health, but not according to the MarketWatch list. Every single city on the list lost jobs, except McAllen, Texas. And where did McAllen rank? No. 83---near the bottom of the list!

And what about taxes? Economic development experts say that low taxes are vital to business health. Cities, counties, and states routinely offer tax exemptions to entice companies to move. So what role did tax rates play in the MarketWatch ratings? None. Tax rates were not included in the 10 criteria. In fact, high-tax cities such as Boston and Washington were in the top 10.

Another common characteristic of the top 10 cities was that the government was the largest employer in at least six of them. Five---Des Moines, Minneapolis/St. Paul, Boise, Denver, and Salt Lake City--are state capitols, with huge state government payrolls. Washington, D.C., is, of course, the quintessential government town.

One more thing: In the MarketWatch survey, size matters. If your city has a Fortune 1000 headquarters, your rank goes up, no matter that Fortune 1000 companies employ less than 5 percent of the American workforce.

Here's another way size matters in the rankings: Only cities with more than 500,000 people are included. That's 101 American cities. No matter that other surveys say the future of American business is the smaller city. Improvements in transportation and communication erase the competitive advantage that cities like New York, Chicago, Boston, and San Francisco had in the 19th and early 20th century---when they had unique access to capital, talent, railheads, ports, and raw materials. Today, a raw material is more likely to be an idea than a railroad car of iron ore, and a "finished good" is more likely to be a software program than a pallet of widgets. Today's goods and services are increasingly likely to go to market over an information superhighway, not a concrete one.

All of this explains why another study said the best cities in America to find a job were (in order) Bismarck, N.D.; Casper, Wyo.; and Logan, Utah. Of the 30 best job-producing cities on the list, only one of them (Salt Lake City) was large enough to make the MarketWatch list.