Limits...
Should methodological filters for diagnostic test accuracy studies be used in systematic reviews of psychometric instruments? A case study involving screening for postnatal depression.

Mann R, Gilbody SM - Syst Rev (2012)

Bottom Line: The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters.Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI).The Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Health Sciences, University of York, York, YO10 5DD, UK. rcm504@york.ac.uk

ABSTRACT

Background: Challenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters. This paper compares the performance of three MEDLINE search strategies for psychometric diagnostic test accuracy (DTA) studies in postnatal depression.

Methods: A reference set of six relevant studies was derived from a forward citation search via Web of Knowledge. The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters. Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI).

Results: The Cochrane recommended strategy and one of the filtered search strategies were equivalent in performance and both retrieved a total of 105 citations, sensitivity was 100% (95% CI 61%, 100%) and precision was 5.2% (2.6%, 11.9%). The second filtered search retrieved a total of 31 citations, sensitivity was 66.6% (30%, 90%) and precision was 12.9% (5.1%, 28.6%). This search missed the DTA study with most relevance to the DTA review.

Conclusions: The Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive. It was considered the optimum method for retrieval of relevant studies for a psychometric DTA review (in this case for postnatal depression). Potential limitations of using filtered searches during a psychometric mental health DTA review should be considered.

Show MeSH

Related in: MedlinePlus

Head to head comparison of three alternative search strategies conducted in Medline 1996 to June, Week 3 2011.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3351740&req=5

Figure 1: Head to head comparison of three alternative search strategies conducted in Medline 1996 to June, Week 3 2011.

Mentions: Table 4 presents the results of the head-to-head comparison of the three strategies. The Cochrane search and CRD filter search each retrieved a total of 105 citations; precision was low (5.2%) due to the number of irrelevant studies identified, however both searches were 100% sensitive and retrieved all reference set studies. The Vincent filter search retrieved a total of 31 citations; although precision was slightly higher due to the smaller number of studies retrieved, the search had poor sensitivity. Of the six papers identified as the reference set, only one paper examined the DTA of the questions compared to a 'gold standard' diagnostic criteria [20], therefore only this study would be eligible for inclusion in a DTA review of the questions. Whilst the Cochrane search and CRD filter search identified this study [20] the Vincent filter search failed to do so. A Venn diagram (Figure 1) shows the commonality and discrepancy between the three searches in MEDLINE.


Should methodological filters for diagnostic test accuracy studies be used in systematic reviews of psychometric instruments? A case study involving screening for postnatal depression.

Mann R, Gilbody SM - Syst Rev (2012)

Head to head comparison of three alternative search strategies conducted in Medline 1996 to June, Week 3 2011.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3351740&req=5

Figure 1: Head to head comparison of three alternative search strategies conducted in Medline 1996 to June, Week 3 2011.
Mentions: Table 4 presents the results of the head-to-head comparison of the three strategies. The Cochrane search and CRD filter search each retrieved a total of 105 citations; precision was low (5.2%) due to the number of irrelevant studies identified, however both searches were 100% sensitive and retrieved all reference set studies. The Vincent filter search retrieved a total of 31 citations; although precision was slightly higher due to the smaller number of studies retrieved, the search had poor sensitivity. Of the six papers identified as the reference set, only one paper examined the DTA of the questions compared to a 'gold standard' diagnostic criteria [20], therefore only this study would be eligible for inclusion in a DTA review of the questions. Whilst the Cochrane search and CRD filter search identified this study [20] the Vincent filter search failed to do so. A Venn diagram (Figure 1) shows the commonality and discrepancy between the three searches in MEDLINE.

Bottom Line: The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters.Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI).The Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Health Sciences, University of York, York, YO10 5DD, UK. rcm504@york.ac.uk

ABSTRACT

Background: Challenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters. This paper compares the performance of three MEDLINE search strategies for psychometric diagnostic test accuracy (DTA) studies in postnatal depression.

Methods: A reference set of six relevant studies was derived from a forward citation search via Web of Knowledge. The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters. Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI).

Results: The Cochrane recommended strategy and one of the filtered search strategies were equivalent in performance and both retrieved a total of 105 citations, sensitivity was 100% (95% CI 61%, 100%) and precision was 5.2% (2.6%, 11.9%). The second filtered search retrieved a total of 31 citations, sensitivity was 66.6% (30%, 90%) and precision was 12.9% (5.1%, 28.6%). This search missed the DTA study with most relevance to the DTA review.

Conclusions: The Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive. It was considered the optimum method for retrieval of relevant studies for a psychometric DTA review (in this case for postnatal depression). Potential limitations of using filtered searches during a psychometric mental health DTA review should be considered.

Show MeSH
Related in: MedlinePlus