A Flexible AI Framework for Census Calibrated Small Domain Prevalence Estimation

Reliable estimation of disease prevalence in small population subgroups is a common challenge in the social sciences. When disease-risk models are available, this issue necessitates reconstructing the joint distribution of covariates, a challenging task when relying solely on survey data. The number of covariate combinations grows rapidly with the number of covariates, making survey data scarce, and the estimated quantities rarely match known marginal quantities available in aggregate form from other sources. We propose a flexible framework to reconstruct joint covariate distributions when survey data are sparse and reliable partial marginals are available from administrative sources. We match the joint covariate distribution when these are known. The missing components of the joint covariate distribution are then estimated sequentially, using highly expressive neural networks trained on survey data. Each covariate is predicted conditionally on those with known joint distributions, or on those previously reconstructed. Reliable marginals are incorporated directly into the loss function through strong penalization, thereby inducing reconstructed distributions to match external totals. We apply the framework to estimate the prevalence of diabetes and cardiovascular disease in fine-grained U.S. subgroups. Combining Behavioral Risk Factor Surveillance System (BRFSS) data with Census marginals, we produce population and subgroup-specific prevalence estimates.