How Many Unlawful Aliens Are in the US?
Published estimates are much less certain than advertised
2 Sep 2010 in Information Quality
Estimating the quantity of something that does not want to be estimated could be the most difficult quantitative task around. For this reason, estimates of the unlawful alien population are prone to both error and bias.
The Pew Hispanic Center published new estimates on September 1. Pew's estimates are reported as being much more certain than they actually are.
The Pew Hispanic Center report (PDF) relies on a complex methodology that, while reported (see Appendix C) is not reproducible by third parties. Taken at face value, Pew's methodology shows that uncertainty in the estimate spans many percentage points. Yet the press release -- what reporters rely on rather than reading the report itself -- gives estimates to the nearest 50,000. This is just one half of one percent of the 11.1 million estimate for March 2009.
Pew claims this is an 8% decline since March 2007. This might be true, but the actual margin of error in its estimates could easily be as large as 8% rather than the reported 0.5%. Another reason for concern is Pew relies on two appeals to authority in lieu of analytical validation: (1) the raw data come from the Census Bureau's Current Population Survey and (2) its methodology it uses is "widely accepted." The appeal to authority is a common logical fallacy.
Federal information quality standards require that uncertainties be propagated all the way to the final result. Pew is not a federal agency, so it is not required to adhere to these principles. However, a federal agency that disseminated Pew's reported estimates would be violating these standards. Moreover, federal agencies cannot rely upon Pew's study even if Pew had accurately reported uncertainty. That's because Pew's methodology is not reproducible, and reproducibility is a minimum federal information quality standard.
