Limits...
Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines.

Kelly SE, Moher D, Clifford TJ - Syst Rev (2016)

Bottom Line: There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.Transparency and inadequate reporting are significant limitations of rapid reviews.Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete.

View Article: PubMed Central - PubMed

Affiliation: School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, H2267A - 40 Ruskin Street, Ottawa, Ontario, Canada. skelly@ottawaheart.ca.

ABSTRACT

Background: Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.

Methods: We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.

Results: Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.

Conclusions: Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.

No MeSH data available.


Related in: MedlinePlus

PRISMA flow diagram
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4862155&req=5

Fig2: PRISMA flow diagram

Mentions: Fourteen HTA agencies responded to the INAHTA scan, and their external web sites were searched for relevant rapid review products following the search for published and unpublished reports. The literature search yielded 5478 titles and abstracts after deduplication across databases. In total, 1008 articles were potentially relevant and their full-text was reviewed. Few rapid review samples in the published literature were located prior to 2011; however, samples of unpublished rapid reviews were plentiful. Following full-text review, 66 rapid reviews produced between 2013 and 2014 fulfilled our eligibility criteria and were included [12–77]. Thirty-three were journal-published [12–44] and 33 were unpublished [45–77] rapid reviews. See Additional file 2 for a full list of included and excluded studies. Figure 2 shows a flow diagram of studies included using guidance from the PRISMA statement [78]. Four hundred unpublished rapid reviews met all other eligibility criteria but were not selected with the majority produced by a small number of agencies who author a high volume of rapid reviews per year e.g. (CADTH). The thirty-three published rapid reviews included were reported in 43 published articles together with companion studies [79–86] for one included rapid review that was published in a journal as a series of 10 articles [19]. We considered these publications a single rapid review as they reported results by intervention from a single literature search. Thirty-one unpublished rapid reviews were located on the websites of their producing agency or organization. Two unpublished rapid reviews [72, 76] were located through contact with primary authors following expert input.Fig. 2


Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines.

Kelly SE, Moher D, Clifford TJ - Syst Rev (2016)

PRISMA flow diagram
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4862155&req=5

Fig2: PRISMA flow diagram
Mentions: Fourteen HTA agencies responded to the INAHTA scan, and their external web sites were searched for relevant rapid review products following the search for published and unpublished reports. The literature search yielded 5478 titles and abstracts after deduplication across databases. In total, 1008 articles were potentially relevant and their full-text was reviewed. Few rapid review samples in the published literature were located prior to 2011; however, samples of unpublished rapid reviews were plentiful. Following full-text review, 66 rapid reviews produced between 2013 and 2014 fulfilled our eligibility criteria and were included [12–77]. Thirty-three were journal-published [12–44] and 33 were unpublished [45–77] rapid reviews. See Additional file 2 for a full list of included and excluded studies. Figure 2 shows a flow diagram of studies included using guidance from the PRISMA statement [78]. Four hundred unpublished rapid reviews met all other eligibility criteria but were not selected with the majority produced by a small number of agencies who author a high volume of rapid reviews per year e.g. (CADTH). The thirty-three published rapid reviews included were reported in 43 published articles together with companion studies [79–86] for one included rapid review that was published in a journal as a series of 10 articles [19]. We considered these publications a single rapid review as they reported results by intervention from a single literature search. Thirty-one unpublished rapid reviews were located on the websites of their producing agency or organization. Two unpublished rapid reviews [72, 76] were located through contact with primary authors following expert input.Fig. 2

Bottom Line: There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.Transparency and inadequate reporting are significant limitations of rapid reviews.Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete.

View Article: PubMed Central - PubMed

Affiliation: School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, H2267A - 40 Ruskin Street, Ottawa, Ontario, Canada. skelly@ottawaheart.ca.

ABSTRACT

Background: Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.

Methods: We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.

Results: Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.

Conclusions: Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.

No MeSH data available.


Related in: MedlinePlus