We have all seen them, lists of the “top ten places for….”. Some are good, others not so much. With our release of the quality of life series, it is inevitable that the data will be used to add to the ever growing list of best and worst places.
We recently were subjected to a top ten list that quite nicely illustrates the problems – “Best Cities For Singles: 10 Places To Rent if You’re Looking for Love”. The winner? Wichita, Kansas. Why? Rents are low, and it has the highest percentage of the population that is single. Not one in a thousand people would have guessed this one correctly.
Not to be overly critical of this particular piece – we have seen many much worse than this – but let’s take a brief look at the elements which are required if you are going to make a good “best” places list.
The Places Must Be Recognizable and Consistent in Size
A ranking of block groups that identifies the top block groups for spelunking would not be overly useful. How many among us know that 350150007002 is Carlsbad Caverns National Park in southeastern New Mexico? Not many, and they all work for either the Census Bureau, AGS, or one of our competitors. So, the areas to be ranked should be named and generally well recognized. This pretty much means states, metropolitan areas, cities, and ZIP codes. Counties can be used, but honestly, how many of us know where Union County is in Pennsylvania? Or that there is a Union County in 17 states plus Union Parish in Louisiana?
Further, the items ranked should not differ too much in size. Comparing median rent between California and Rhode Island is not particularly meaningful as one is nearly forty times the size of the other. The larger the area, the more likely its statistics will trend towards the national average. We should be looking for relatively small, consistent sized, and recognizable areas. ZIP codes are probably the most consistent, but if you are going to use them, be sure to include the post office name so that users at least have a sense of where it might be.
The remedy is to narrow lists down to things like “The Top Ten Mid-Size Cities for…” so that at least we are attempting to compare similar items. The narrower the definition, the more likely that the list is going to be meaningful.
Know What You Are Measuring
This is probably the most important part, and the one where many lists fall apart. In this case, we have two essential elements – the single population and rent. Why rent? Beats us, as it assumes that we are looking not for singles in general but rather singles with no money.
What do we mean by single? Do we mean ‘never married’? Or are we including ‘now separated’, ‘widowed’, and ‘divorced’ in the total? This is not defined for us, so we don’t know. If you want a broad definition of singles, go to Gallup NM, where the separated, widowed, and divorced population outnumbers the currently married population.
Are we measuring the concentration of singles? Or the sheer number? If we are measuring the concentration (%), we are likely to find high concentrations in smaller places simply because of the laws of central tendency. If we are looking for volume, the larger places will certainly be at the top of the list. Most of these lists involve looking at multiple variables which must be given relative weights in the analysis. We can only guess at the weights used in the singles list – if the percent single is dominant, then we are really saying that we are looking for anybody who is single, but if the average rent is dominant, then we are saying that the person looking for singles is cheap.
It must be understood that variables which seem related at the individual level (single and renting) are translated well into geographic area summaries. To really rank cities using single and low rent as the target, you would need to have a joint distribution of marital status and rent. We could produce an effective ranking of households who are both young (say age <35) and affluent (say income > $100,000) but in most cases, there is a significant issue with what is known as the ecological correlation problem.
The Recipe for a Meaningful Ranking
A meaningful ranking will always have the following:
- The places ranked are generally recognizable and of relatively consistent size
- The variables chosen for ranking will have a clear relationship to the purpose of the list and will be clearly stated
- The ranking methodology will include at least a cursory statement of what was considered important, and in what relative quantity
If your ranking produces a surprise winner that no one would have ever guessed, you probably don’t have a meaningful ranking. You might have good click-bait, but that’s about all.
Recent Comments