Limits...
Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines.

Kelly SE, Moher D, Clifford TJ - Syst Rev (2016)

Bottom Line: There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.Transparency and inadequate reporting are significant limitations of rapid reviews.Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete.

View Article: PubMed Central - PubMed

Affiliation: School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, H2267A - 40 Ruskin Street, Ottawa, Ontario, Canada. skelly@ottawaheart.ca.

ABSTRACT

Background: Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.

Methods: We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.

Results: Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.

Conclusions: Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.

No MeSH data available.


Related in: MedlinePlus

Star chart depicting proportions of rapid reviews meeting PRISMA reporting guidelines by item (n = 66)
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4862155&req=5

Fig5: Star chart depicting proportions of rapid reviews meeting PRISMA reporting guidelines by item (n = 66)

Mentions: The mean number of adequately reported PRISMA items was 13.2 (SD 6.0) out of the maximum possible 27. Items were adequately reported 49 % of the time on average in the 66 included rapid review samples. Figure 5 shows the proportion of rapid reviews that adequately reported the individual PRISMA checklist items. Individual items that were reported well in a large proportion of rapid reviews were the following: describing all information sources in the search (item 7, 81 %), presenting the main results of the review in a synthesis of results (item 21, 88 %), summarizing the main findings with relevance to key groups (item 24, 74 %) and providing general interpretation and context for the results of the review in the conclusions (item 26, 89 %). Other items were very poorly reported, such as indicating if a protocol exists or is registered (item 5, 6 %), describing the process of data collection (item 10, 30 %) and discussing the study limitations at the study/outcome and review level (item 25, 40 %). Less than 50 % of rapid reviews described the methods used for assessing risk of bias in individual studies (item 12, 44 %) or presented data on the risk of bias in each study (item 19, 48 %).Fig. 5


Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines.

Kelly SE, Moher D, Clifford TJ - Syst Rev (2016)

Star chart depicting proportions of rapid reviews meeting PRISMA reporting guidelines by item (n = 66)
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4862155&req=5

Fig5: Star chart depicting proportions of rapid reviews meeting PRISMA reporting guidelines by item (n = 66)
Mentions: The mean number of adequately reported PRISMA items was 13.2 (SD 6.0) out of the maximum possible 27. Items were adequately reported 49 % of the time on average in the 66 included rapid review samples. Figure 5 shows the proportion of rapid reviews that adequately reported the individual PRISMA checklist items. Individual items that were reported well in a large proportion of rapid reviews were the following: describing all information sources in the search (item 7, 81 %), presenting the main results of the review in a synthesis of results (item 21, 88 %), summarizing the main findings with relevance to key groups (item 24, 74 %) and providing general interpretation and context for the results of the review in the conclusions (item 26, 89 %). Other items were very poorly reported, such as indicating if a protocol exists or is registered (item 5, 6 %), describing the process of data collection (item 10, 30 %) and discussing the study limitations at the study/outcome and review level (item 25, 40 %). Less than 50 % of rapid reviews described the methods used for assessing risk of bias in individual studies (item 12, 44 %) or presented data on the risk of bias in each study (item 19, 48 %).Fig. 5

Bottom Line: There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.Transparency and inadequate reporting are significant limitations of rapid reviews.Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete.

View Article: PubMed Central - PubMed

Affiliation: School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, H2267A - 40 Ruskin Street, Ottawa, Ontario, Canada. skelly@ottawaheart.ca.

ABSTRACT

Background: Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.

Methods: We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.

Results: Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.

Conclusions: Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.

No MeSH data available.


Related in: MedlinePlus