Limits...
A Meta-Analysis of Typhoid Diagnostic Accuracy Studies: A Recommendation to Adopt a Standardized Composite Reference.

Storey HL, Huang Y, Crudder C, Golden A, de los Santos T, Hawkins K - PLoS ONE (2015)

Bottom Line: Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard.In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed.We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality.

View Article: PubMed Central - PubMed

Affiliation: Diagnostics Program, PATH, Seattle, Washington, United States of America.

ABSTRACT
Novel typhoid diagnostics currently under development have the potential to improve clinical care, surveillance, and the disease burden estimates that support vaccine introduction. Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard. If no single gold standard test exists, use of a composite reference standard (CRS) can improve estimation of diagnostic accuracy. Numerous studies have used a CRS to evaluate new typhoid diagnostics; however, there is no consensus on an appropriate CRS. In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed. We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality. This wide-ranging meta-analysis suggests that no single test has sufficiently good performance but some existing diagnostics may be useful as part of a CRS. Additionally, based on findings from the meta-analysis and a constructed numerical example demonstrating the use of CRS, we proposed necessary criteria and potential components of a typhoid CRS to guide future recommendations. Agreement and adoption by all investigators of a standardized CRS is requisite, and would improve comparison of new diagnostics across independent studies, leading to the identification of a better reference test and improved confidence in prevalence estimates.

Show MeSH

Related in: MedlinePlus

Comparisons by reference test.Summary of the 139 papers by reference test, including 413 index/reference comparisons. Of the culture reference tests, 80% were blood culture, making up 57% of all reference tests.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4643909&req=5

pone.0142364.g002: Comparisons by reference test.Summary of the 139 papers by reference test, including 413 index/reference comparisons. Of the culture reference tests, 80% were blood culture, making up 57% of all reference tests.

Mentions: All index/reference combinations were first summarized by type of reference standard used (Fig 2). Index tests were then categorized into five broad groups, based on the detection of nucleic acid, antigen, antibody, viable bacteria, or clinical features (including composite tests). Diagnostic accuracies of individual index tests were analyzed using meta-analysis when particular index/reference combinations were present in more than five comparisons, to increase the relevance of the conclusions. The majority of meta-analyses were based on combinations using blood culture as the reference test. Statistical analyses were performed using STATA® 11.2 (StataCorp, TX, USA) and R version 3.1.3 (https://www.r-project.org/). Summary results were calculated in the meta-analysis using bivariate random effects binomial regression (STATA command: metandi), and listed in the text as follows: summary sensitivity (Sens) (95% Confidence Interval, CI), summary specificity (Spec) (95% CI), and summary diagnostic odds ratio (DOR) (95% CI) [183,184]. Sources of heterogeneity in the summary estimates were investigated in the meta-regression analysis, also using bivariate random effects binomial regression (STATA command: midas) [185]. Potential covariates were transformed into dichotomous variables and evaluated as appropriate for each index test (Table 1) [182]. The hierarchical summary receiver operating characteristic curves were also generated [186].


A Meta-Analysis of Typhoid Diagnostic Accuracy Studies: A Recommendation to Adopt a Standardized Composite Reference.

Storey HL, Huang Y, Crudder C, Golden A, de los Santos T, Hawkins K - PLoS ONE (2015)

Comparisons by reference test.Summary of the 139 papers by reference test, including 413 index/reference comparisons. Of the culture reference tests, 80% were blood culture, making up 57% of all reference tests.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4643909&req=5

pone.0142364.g002: Comparisons by reference test.Summary of the 139 papers by reference test, including 413 index/reference comparisons. Of the culture reference tests, 80% were blood culture, making up 57% of all reference tests.
Mentions: All index/reference combinations were first summarized by type of reference standard used (Fig 2). Index tests were then categorized into five broad groups, based on the detection of nucleic acid, antigen, antibody, viable bacteria, or clinical features (including composite tests). Diagnostic accuracies of individual index tests were analyzed using meta-analysis when particular index/reference combinations were present in more than five comparisons, to increase the relevance of the conclusions. The majority of meta-analyses were based on combinations using blood culture as the reference test. Statistical analyses were performed using STATA® 11.2 (StataCorp, TX, USA) and R version 3.1.3 (https://www.r-project.org/). Summary results were calculated in the meta-analysis using bivariate random effects binomial regression (STATA command: metandi), and listed in the text as follows: summary sensitivity (Sens) (95% Confidence Interval, CI), summary specificity (Spec) (95% CI), and summary diagnostic odds ratio (DOR) (95% CI) [183,184]. Sources of heterogeneity in the summary estimates were investigated in the meta-regression analysis, also using bivariate random effects binomial regression (STATA command: midas) [185]. Potential covariates were transformed into dichotomous variables and evaluated as appropriate for each index test (Table 1) [182]. The hierarchical summary receiver operating characteristic curves were also generated [186].

Bottom Line: Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard.In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed.We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality.

View Article: PubMed Central - PubMed

Affiliation: Diagnostics Program, PATH, Seattle, Washington, United States of America.

ABSTRACT
Novel typhoid diagnostics currently under development have the potential to improve clinical care, surveillance, and the disease burden estimates that support vaccine introduction. Blood culture is most often used as the reference method to evaluate the accuracy of new typhoid tests; however, it is recognized to be an imperfect gold standard. If no single gold standard test exists, use of a composite reference standard (CRS) can improve estimation of diagnostic accuracy. Numerous studies have used a CRS to evaluate new typhoid diagnostics; however, there is no consensus on an appropriate CRS. In order to evaluate existing tests for use as a reference test or inclusion in a CRS, we performed a systematic review of the typhoid literature to include all index/reference test combinations observed. We described the landscape of comparisons performed, showed results of a meta-analysis on the accuracy of the more common combinations, and evaluated sources of variability based on study quality. This wide-ranging meta-analysis suggests that no single test has sufficiently good performance but some existing diagnostics may be useful as part of a CRS. Additionally, based on findings from the meta-analysis and a constructed numerical example demonstrating the use of CRS, we proposed necessary criteria and potential components of a typhoid CRS to guide future recommendations. Agreement and adoption by all investigators of a standardized CRS is requisite, and would improve comparison of new diagnostics across independent studies, leading to the identification of a better reference test and improved confidence in prevalence estimates.

Show MeSH
Related in: MedlinePlus