Error and bias in under-5 mortality estimates derived from birth histories with small sample sizes

Abstract

Under-5 mortality, the probability of death before age 5, is an important indicator of child health in a population. Because estimates of under-5 mortality are often derived from birth history data from censuses or surveys, it is important to know how accurate these estimates are, particularly estimates derived from small samples of women. Researchers aimed to assess the magnitude and direction of error for estimates derived from birth histories using several analysis methods.

Analytical approach

Birth history data from 152 Demographic and Health Surveys in 62 countries were treated as populations with known under-5 mortality. Different sample sizes (10, 50, 100, 500, and 1,000) were drawn from these populations to mimic surveys with small sample sizes. Summary birth history methods and complete birth history methods were applied to the data in each sample, and the resulting under-5 mortality estimates were compared to the true under-5 mortality in the population.

Findings

The researchers found that all methods studied are prone to high levels of error at the smallest sample size. In addition, for many methods, the amount of error differed by the level of true mortality or by the amount of time prior to survey data collection.

Conclusions

The authors conclude that since there is no clear best method to choose for analyzing birth history data from small sample sizes, these results suggest that birth histories from all but the largest of surveys are of limited utility for estimating under-5 mortality at the subnational level. Further research into methods for using existing data sources, including small area estimation methods, and investment in alternative data sources are warranted in countries without high-quality vital registration systems.