Limits...
PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

Armijo-Olivo S, da Costa BR, Cummings GG, Ha C, Fuentes J, Saltaji H, Egger M - PLoS ONE (2015)

Bottom Line: Meta-epidemiological study within Cochrane Database of Systematic Reviews.Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32).When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6.

View Article: PubMed Central - PubMed

Affiliation: CLEAR (Connecting Leadership, Education, and Research) Outcomes Research Program, University of Alberta, Faculty of Nursing, University of Alberta, Edmonton, Canada; Rehabilitation Research Center, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, Canada.

ABSTRACT

Objective: There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria.

Design: Meta-epidemiological study within Cochrane Database of Systematic Reviews.

Methods: Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis.

Results: Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6.

Conclusions: The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

No MeSH data available.


Agreement between PeDro Score at different cut offs and Cochrane Approach.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4498768&req=5

pone.0132634.g002: Agreement between PeDro Score at different cut offs and Cochrane Approach.

Mentions: Agreement between PeDro and Cochrane for the definition of adequate quality across all meta-analyses was poor for PeDro scores >5 or more (kappa 0.12; 95% CI 0.07–0.16), slight for a score >6 or more (kappa 0.24; 95% CI 0.16–0.32), and 7 or more (kappa 0.39; 95% CI 0.286–0.510), and fair (kappa 0.44; 95% CI 0.314–0.574) for 8 points and more (Fig 2).


PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

Armijo-Olivo S, da Costa BR, Cummings GG, Ha C, Fuentes J, Saltaji H, Egger M - PLoS ONE (2015)

Agreement between PeDro Score at different cut offs and Cochrane Approach.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4498768&req=5

pone.0132634.g002: Agreement between PeDro Score at different cut offs and Cochrane Approach.
Mentions: Agreement between PeDro and Cochrane for the definition of adequate quality across all meta-analyses was poor for PeDro scores >5 or more (kappa 0.12; 95% CI 0.07–0.16), slight for a score >6 or more (kappa 0.24; 95% CI 0.16–0.32), and 7 or more (kappa 0.39; 95% CI 0.286–0.510), and fair (kappa 0.44; 95% CI 0.314–0.574) for 8 points and more (Fig 2).

Bottom Line: Meta-epidemiological study within Cochrane Database of Systematic Reviews.Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32).When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6.

View Article: PubMed Central - PubMed

Affiliation: CLEAR (Connecting Leadership, Education, and Research) Outcomes Research Program, University of Alberta, Faculty of Nursing, University of Alberta, Edmonton, Canada; Rehabilitation Research Center, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, Canada.

ABSTRACT

Objective: There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria.

Design: Meta-epidemiological study within Cochrane Database of Systematic Reviews.

Methods: Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis.

Results: Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6.

Conclusions: The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

No MeSH data available.