Last week we introduced our Vortex demographics engine which will revolutionize small area demographic forecasting. While we will get into the details of the engine next week, the key to its performance is data.
Think of Vortex like a massive centrifuge.
Individual data sources, no matter how good they might be, will contain some degree of error. Further, the data may have been collected for a completely different purpose and may have significant problems in other use cases. A great example of this is the postal service database that tracks new ZIP+4 creation, about which we have written considerably in the past. Obviously, the postal service has to deliver mail to a new residential tract, but at a local level, the creation of a ZIP+4 does not mean that there are new houses – they are likely in the early stages of planning and will in some cases never come to fruition. A ZIP+4 which never materializes is not an error at the postal service, but if used to project growth will result in massive local projection error.
The solution to the problem is to use as many different data sources as possible, which allows you to recognize that error in each of the data sources is simply noise within a complex system. And the more sources you employ, the better you can separate noise from real signals. If you mix many chemicals together, then run them through a centrifuge, the noise will be separated from the substance you are trying to isolate.
The data sources we are using are, when possible, time series on a consistent geographic base (H3 hexagons, level 8). We are adding new data sources with every iteration of the model, so this list will grow over time:
- Core annual demographic estimates from AGS dating back to 2017
- Census enumerations for 1990, 2000, 2010, and 2020
- Quarterly national parcel file tracking change in –
- Land use
- Ownership, especially purchases by corporate entities
- Parcel geometry change (amalgamations, subdivision)
- Quarterly building permits
- Points of interest by category, quarterly
- HMDA mortgage applications, annual
- FEMA disaster tracking and housing damage estimates, from 2002 to the present
- Annual changes to the road network, flagging newly constructed segments and classification changes to existing segments
- HUD assisted multi-family housing locations, annual time series
- Census Bureau LODES database, tracking journey to work flows
- AGS Crime Risk
- NCES school database
- Traffic counts
- Census Bureau block level housing unit estimates (semi-annual)
- Sales pages for major residential builders (the top 250)
The model is also subject to controls at higher levels – national and regional population estimates, fertility and mortality models, and estimates of net immigration. These ensure that the models don’t stray beyond the expected guardrails.
One of the key features of the Vortex is that it is a spatial-time series model, meaning that each hexagon is treated as part of a spatial system and not as an independent observation unit. As part of a metropolitan area, growth signals in one census block may affect nearby blocks where there are no signals. The city is treated as a growing, spilling into adjacent areas. As such, the model needs controls on how much development is possible, and this includes –
- Existing building footprints and vacant land estimates
- Land use
- Elevation and terrain limitations
- Developmentally protected areas, such as parks and wetlands
- Accessibility measures (distances to airports, freeway interchanges, rail stations, fixed transit lines, bus stops and bus lines)
Next week, we will talk about how the model ingests and interprets the wide variety of data which is being used and how it works as a centrifuge in the presence of mixed signals and error.