The decennial census of the United States is a constitutionally mandated function of the federal government, the main purpose of which is the apportionment of representatives. Over time, the scope of the census questionnaire was generally expanded, and in 1940, the “long form” was introduced on a sampled basis. This form was used to collect detailed demographic and housing attributes, but likely resulted in higher non-response rates. By the 1990’s, many users of data were clamoring for more frequently updated data than the decennial census.
The result was the American Community Survey (ACS), which was first undertaken in 2000 to overlap with the census, and it entirely replaced the long form for the 2010 and 2020 censuses. The ACS is a continuous survey, reaching about 250,000 households every month and released annually. Five years of data are required in order to release data at the block group level.
The ACS suffers from response rate problems, which have tended in recent years to be in the low 80% zone, but dropped substantially in 2020 due to the pandemic, especially in group quarters where response rates were below 50%. Response rates have been generally declining since the inception of the survey, which in the early years obtained response rates of well over 95%.
The sampling frame for the ACS is intended to provide estimates for every block group over a five-year period. Sampling is carried out at the block level, with blocks divided into one of sixteen sampling buckets based on the block, census tract, and county size. For the 2019 survey, sampling rates ranged from 15% in small blocks in small counties to 0.5% in large blocks in large counties.
The sample is primarily drawn in five-year intervals using the latest release of the MAF (Master Address File). Since the ACS rules are that no address be sampled more than once every five years, the addresses are split into five annual sample buckets. Over the lifespan of the sample, new addresses are equally distributed into the five groups as if they existed at the start of the sample frame. While this makes sense from a perspective of tracking the demographics of a census block, it likely means greater error in housing related estimates – especially if new development is different than the existing stock (e.g. a townhouse development within a block with existing single-family homes, or the removal of single-family homes and construction of a multi-family dwelling).
Users of ACS data should be aware that, unlike most statistical analyses, confidence limits are set at 90% not the usual 95%. The result is that many estimates, even for large areas, have very broad error bands. For example, for Thousand Oaks, CA (population about 126,000), the following estimates of population by age are found in the latest 1 year and 5-year releases:
Note that the estimate of the 5-year ACS (2018-2022) for females aged 5-9 should be interpreted as “we are 90% confident that the actual value is between 2,812 and 3,710”. As large as this range is, compare how radically different the 2022 and 2023 one year estimates are, and how much they differ from the two recent 5-year estimate. Note that the differences between the 2022 and 2023 ACS 1-year estimates are well within the standard error, but that clearly these should not be used for time series analysis. What makes matters worse is the extent to which data are “allocated”. For many detailed attributes, this can be quite high – for example in the latest 5-year ACS nearly 20% of the record in the Means of Transportation to Work for the city of Thousand Oaks were allocated.
We believe that the sampling method works best in stable areas, but results in significant error in areas of substantial growth or redevelopment. For most users of small area demographics, these are typically the areas of most interest.
Given the budgetary constraints on the program, we do not foresee major revisions to the ACS program, which would require a substantial increase in sample size to yield more precise results. It is worth noting that the budget for the ACS in 2023 was $233.5 million, which still isn’t enough to increase the sample size.
It has become clear to AGS in recent years that we must take major steps to improve upon the standard data sources which have been foundational to small area demographics. One of those steps is to utilize national parcel records, and especially the changes to those parcels over time, to bolster the housing characteristics. The initial focus is to improve the dwelling estimates which includes not only the total dwelling count, but dwellings by age, size and type. By comparing the parcel address and the mailing address for taxation, we can identify houses which are rented and track changes long before they are detectable in the ACS.
Recent Comments