Just a few weeks ago, the moderator in a presidential debate ‘fact checked’ one candidate’s assertion that crime rates were going up in the United States, citing the 2022 Uniform Crime Report (UCR) which said the opposite. The news recently hit that upon the release of the 2023 report, the FBI without comment had revised its numbers for both 2021 and 2022. What had been touted as a decline in crime turned out to be a significant increase.
The Crime Prevention Research Center released a report entitled “The Revised FBI Crime Data Reveals that it Originally Missed 1,699 Murders in 2022. Given that Almost all Murders are Reported, How Does the FBI Miss that Many Murders? (crimeresearch.org)”. As they point out, while many lesser crimes go unreported, murder is not one of them.
In some respects, it is understandable that there were issues with the data. The FBI implemented its new National Incident-Based Reporting System (NIBRS) to provide national statistics on an expanded number of crimes, replacing a system that had been in place since 1930. The 2021 report was the first on the new system and quite predictably, some agencies did not manage to implement changes despite having well over a decade to prepare. Some minor cities like Los Angeles, New York, Philadelphia, Baltimore. Nothing to be concerned about. Florida? No data. California? A whopping eight agencies managed the transition.
The FBI statistics employees therefore employed some unspecified methods to fill in the missing data for both 2021 and 2022. To be fair, those methods have likely been used for many decades, since every year there are bound to be some jurisdictions who fail to report in what is, after all, a voluntary reporting system.
To those wondering about the AGS CrimeRisk product, we were certainly aware of these issues and took steps to correct them. The 2021 UCR numbers were highly suspicious – the top line state estimates that get news headlines were actually lower than the computed total for the jurisdictions in those states that actually reported. The FBI analysts surely knew this and yet released the state and national totals anyways.
Fortunately, we collect incident data for a now substantial number of large cities – including many that did not report in 2021, which allowed us to adjust the FBI numbers using local data instead of relying on the FBI’s unspecified adjustments. The 2022 UCR report was better, but we again made adjustments. Since we use a seven-year rolling model, the impacts of the FBI data quality issues were minimized but certainly not eliminated.
But we do have questions. Given the major gaps in the data, why did the FBI not do something simple, obvious, and relatively inexpensive to provide more accurate estimates? Those non-reporting agencies didn’t stop tracking crime, so why not assign a couple of staff to get those numbers by making some telephone calls? The NIBRS is updated regularly by most agencies, so the FBI knew well in advance that the data was going to be highly suspect. And they certainly have the contact information at each of these jurisdictions. Instead, they used unspecified estimation methods and changed the numbers without comment. This may be understandable but not when those adjustments completely change the narrative, so we shouldn’t be surprised that the FBI is now being accused of having political motivations.
Other federal statistical agencies have also been accused of playing politics with data – the BLS monthly job estimates are widely mocked because they are consistently and quietly revised downward a few months after release. Only recently has this become news because someone actually tabulated the differences over a calendar year between the monthly releases and their subsequent revisions. Likewise, since changes in Social Security are tied to the inflation numbers, we are rarely surprised that the reported CPI changes often don’t align with our personal experiences.
Political influence in the analysis and reporting of key social and economic statistics must never be permitted in a free and open democratic society. Trust in the system is greatly damaged when the daily experiences of the people don’t align with the statistical story they are being told – are you going to believe the government? Or your own lying eyes? These statistics help shape both private decisions and public policy – and often influence elections – and bad data generally leads to bad decisions.
We do not know if the FBI estimates for 2021 and 2022 were politically influenced, and we are not saying that they were. But what is clear is that the lack of transparency is sufficient to merit asking the question.
Recent Comments