Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets

Most developing countries lack complete vital registration systems that gather detailed information on mortality and causes of death. In these settings, VA methods are critically important for evaluating the leading causes of death and ultimately measuring the burden of disease, injuries, and risk factors. Researchers currently use a variety of VA methods to collect and analyze data.

 

To help evaluate the performance and accuracy of these methods in assigning a cause of death, PHMRC set out to create a high-quality validation dataset from different populations. This required finding deaths in low-resource settings, including Mexico, India, Tanzania, and the Philippines, that could be verified through robust gold standard (GS) criteria, which included clinical endpoints, laboratory findings, medical imaging, and pathology. This study is part of ongoing work by IHME to develop the most accurate and efficient methods of using VA to determine individual-  and population-level causes of death.

PHMRC collected 12,542 VAs on deaths and attempted to verify them through the GS criteria. This included 7,836 adults, 2,075 children, 1,629 neonates, and 1,002 stillbirths. The majority of the deaths met the highest level of GS criteria. Overall, 86% of adult deaths, 81% of child deaths, and 99.7% of neonate deaths met the highest level of GS diagnostic criteria. The research revealed the low quality of medical records and diagnosis in some countries, underscoring the importance of looking beyond hospital records for assigning causes of death.

The PHMRC worked in six sites in four countries: Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India. Researchers developed strict diagnostic criteria that included laboratory, pathology, and medical imaging findings to identify gold standard deaths in health facilities.

 

Researchers also developed an enhanced VA instrument based on World Health Organization (WHO) standards. To build the cause of death list, researchers took into account the WHO Global Burden of Disease estimates of the leading causes of death, the potential to identify unique signs and symptoms, and the likely existence of sufficient medical technology to ascertain gold standard cases.

The PHMRC believes that this robust validation dataset will allow researchers to evaluate the performance of different VA analytic methods as well as instrument design. This dataset can be used to inform the implementation of VAs to more reliably ascertain cause of death in national health information systems. To meet urgent policy and planning needs, countries need to introduce VA methods to gather more information on the leading causes of death. It is critical that they be able to test the performance and validity of the methods they are currently using and any new methods they might consider using in the future.

 

They need to be able to assess these methods against rigorous, standardized criteria that are not influenced by the often poor quality of medical records or the diagnostic biases of physicians. Collecting a comparable GS dataset would be prohibitively expensive, meaning that the PHMRC dataset likely will remain the largest validation set for years to come. The PHMRC intends to make an anonymized dataset publicly available as a resource for the broader VA scientific community interested in developing and testing new methods.