The Impact of Covariance on American Community Survey Margins of Error: Computational Alternatives

The American Community Survey (ACS) is an indispensable tool for studying the United States (US) population. Each year the US Census Bureau (BOC) publishes approximately 11 billion ACS estimates, each of which is accompanied by a margin of error (MOE) specific to that estimate. Researchers, policy makers, and government agencies combine these estimates in myriad ways, which requires an accurate measurement of the MOE on that combined estimate. We compare three options for computing this MOE: the analytic approach uses standard statistically derived formulas, the simulation approach builds an empirical distribution of the combined estimate based on simulated values of the inputs, and the replicate approach uses simulated values published by the BOC based on their internal model that statistically replicates the entire ACS 80 times. We find that since the replicate approach is the only one of the three to incorporate covariance between the input variables, it performs the best. We further find that the simulation and analytic approaches generally match one another and can both overestimate and underestimate the MOE; they have their places when the replicate approach is not feasible.