Limits...
Constraint-induced movement therapy: trial sequential analysis applied to Cochrane collaboration systematic review results.

Castellini G, Gianola S, Banzi R, Corbetta D, Gatti R, Sirtori V, Gluud C, Moja L - Trials (2014)

Bottom Line: We calculated the information size and the cumulative Z-statistic, applying the O'Brien-Fleming monitoring boundaries.We included data from 14 RCTs.Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

View Article: PubMed Central - PubMed

Affiliation: Health Professional Science of Rehabilitation, University of Milan, Via Festa del Perdono, 7, 20122 Milan, Italy. gre.caste@gmail.com.

ABSTRACT

Background: Trial sequential analysis (TSA) may establish when firm evidence about the efficacy of interventions is reached in a cumulative meta-analysis, combining a required information size with adjusted thresholds for conservative statistical significance. Our aim was to demonstrate TSA results on randomized controlled trials (RCTs) included in a Cochrane systematic review on the effectiveness of constraint-induced movement therapy (CIMT) for stroke patients.

Methods: We extracted data on the functional independence measure (FIM) and the action research arm test (ARAT) from RCTs that compared CIMT versus other rehabilitative techniques. Mean differences (MD) were analyzed using a random-effects model. We calculated the information size and the cumulative Z-statistic, applying the O'Brien-Fleming monitoring boundaries.

Results: We included data from 14 RCTs. In the conventional meta-analysis (seven trials, 233 patients), the effect of CIMT on FIM was reported as significant (MD 2.88, 95% CI 0.08 to 5.68; P = 0.04). The diversity-adjusted required information size was 142 patients, and the cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.02 to 5.78). The effect of CIMT on ARAT (nine trials, 199 patients) was reported as significant (MD 7.78, 95% CI 1.19 to 14.37; P = 0.02). However, the diversity-adjusted required information size was 252 patients, and the Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).

Conclusions: Although conventional meta-analyses of CIMT reached statistical significance, their overall results remain inconclusive and might be spurious. Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

Show MeSH

Related in: MedlinePlus

Trial sequential analysis constraint-induced movement therapy (CIMT) versus control for arm motor function (ARAT). Diversity-adjusted required information size of 252 patients calculated on basis of 12 MID of ARAT; a variance of 98; alpha = 5%; beta = 20%, and a diversity (D or rather D2 of 91%). The blue cumulative Z-curve crosses first the horizontal green line of conventional statistical significance (alpha = 0.05) during the second trial and then crossed the inward sloping red trial sequential monitoring boundary for benefit during the fifth trial. After the ninth trial, the cumulative Z-score for 199 patients of 2.31 is just below the trial sequential monitoring boundary score of 2.33. With 199 out of 252 (79.0%) patients randomized we still have insufficient evidence to accept or reject an effect of CIMT on ARAT, although it looked promising during the fifth, sixth, and seventh trials (disregarding the risks of bias). The analysis and the figure were made with the TSA software. Legend. Square symbol: z-score for single study; Diamond symbol: trial sequential monitoring boundary for benefit score for single study. Abbreviations: DARIS, diversity-adjusted required information size; MID, minimal important difference; var, variance; D, diversity; a, alpha; b, beta.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4307139&req=5

Fig5: Trial sequential analysis constraint-induced movement therapy (CIMT) versus control for arm motor function (ARAT). Diversity-adjusted required information size of 252 patients calculated on basis of 12 MID of ARAT; a variance of 98; alpha = 5%; beta = 20%, and a diversity (D or rather D2 of 91%). The blue cumulative Z-curve crosses first the horizontal green line of conventional statistical significance (alpha = 0.05) during the second trial and then crossed the inward sloping red trial sequential monitoring boundary for benefit during the fifth trial. After the ninth trial, the cumulative Z-score for 199 patients of 2.31 is just below the trial sequential monitoring boundary score of 2.33. With 199 out of 252 (79.0%) patients randomized we still have insufficient evidence to accept or reject an effect of CIMT on ARAT, although it looked promising during the fifth, sixth, and seventh trials (disregarding the risks of bias). The analysis and the figure were made with the TSA software. Legend. Square symbol: z-score for single study; Diamond symbol: trial sequential monitoring boundary for benefit score for single study. Abbreviations: DARIS, diversity-adjusted required information size; MID, minimal important difference; var, variance; D, diversity; a, alpha; b, beta.

Mentions: An anchor-based approach was used to determine the MID in the RCT by Lang et al. [56]. The estimated MID was 12 points, corresponding to the diversity-adjusted required information size of 252 patients (Figure 5). If the risks of bias are disregarded, the cumulative Z-curve crossed the trial sequential monitoring boundaries after the fifth trial, demonstrating a significant intervention effect. However, the final cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).Figure 5


Constraint-induced movement therapy: trial sequential analysis applied to Cochrane collaboration systematic review results.

Castellini G, Gianola S, Banzi R, Corbetta D, Gatti R, Sirtori V, Gluud C, Moja L - Trials (2014)

Trial sequential analysis constraint-induced movement therapy (CIMT) versus control for arm motor function (ARAT). Diversity-adjusted required information size of 252 patients calculated on basis of 12 MID of ARAT; a variance of 98; alpha = 5%; beta = 20%, and a diversity (D or rather D2 of 91%). The blue cumulative Z-curve crosses first the horizontal green line of conventional statistical significance (alpha = 0.05) during the second trial and then crossed the inward sloping red trial sequential monitoring boundary for benefit during the fifth trial. After the ninth trial, the cumulative Z-score for 199 patients of 2.31 is just below the trial sequential monitoring boundary score of 2.33. With 199 out of 252 (79.0%) patients randomized we still have insufficient evidence to accept or reject an effect of CIMT on ARAT, although it looked promising during the fifth, sixth, and seventh trials (disregarding the risks of bias). The analysis and the figure were made with the TSA software. Legend. Square symbol: z-score for single study; Diamond symbol: trial sequential monitoring boundary for benefit score for single study. Abbreviations: DARIS, diversity-adjusted required information size; MID, minimal important difference; var, variance; D, diversity; a, alpha; b, beta.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4307139&req=5

Fig5: Trial sequential analysis constraint-induced movement therapy (CIMT) versus control for arm motor function (ARAT). Diversity-adjusted required information size of 252 patients calculated on basis of 12 MID of ARAT; a variance of 98; alpha = 5%; beta = 20%, and a diversity (D or rather D2 of 91%). The blue cumulative Z-curve crosses first the horizontal green line of conventional statistical significance (alpha = 0.05) during the second trial and then crossed the inward sloping red trial sequential monitoring boundary for benefit during the fifth trial. After the ninth trial, the cumulative Z-score for 199 patients of 2.31 is just below the trial sequential monitoring boundary score of 2.33. With 199 out of 252 (79.0%) patients randomized we still have insufficient evidence to accept or reject an effect of CIMT on ARAT, although it looked promising during the fifth, sixth, and seventh trials (disregarding the risks of bias). The analysis and the figure were made with the TSA software. Legend. Square symbol: z-score for single study; Diamond symbol: trial sequential monitoring boundary for benefit score for single study. Abbreviations: DARIS, diversity-adjusted required information size; MID, minimal important difference; var, variance; D, diversity; a, alpha; b, beta.
Mentions: An anchor-based approach was used to determine the MID in the RCT by Lang et al. [56]. The estimated MID was 12 points, corresponding to the diversity-adjusted required information size of 252 patients (Figure 5). If the risks of bias are disregarded, the cumulative Z-curve crossed the trial sequential monitoring boundaries after the fifth trial, demonstrating a significant intervention effect. However, the final cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).Figure 5

Bottom Line: We calculated the information size and the cumulative Z-statistic, applying the O'Brien-Fleming monitoring boundaries.We included data from 14 RCTs.Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

View Article: PubMed Central - PubMed

Affiliation: Health Professional Science of Rehabilitation, University of Milan, Via Festa del Perdono, 7, 20122 Milan, Italy. gre.caste@gmail.com.

ABSTRACT

Background: Trial sequential analysis (TSA) may establish when firm evidence about the efficacy of interventions is reached in a cumulative meta-analysis, combining a required information size with adjusted thresholds for conservative statistical significance. Our aim was to demonstrate TSA results on randomized controlled trials (RCTs) included in a Cochrane systematic review on the effectiveness of constraint-induced movement therapy (CIMT) for stroke patients.

Methods: We extracted data on the functional independence measure (FIM) and the action research arm test (ARAT) from RCTs that compared CIMT versus other rehabilitative techniques. Mean differences (MD) were analyzed using a random-effects model. We calculated the information size and the cumulative Z-statistic, applying the O'Brien-Fleming monitoring boundaries.

Results: We included data from 14 RCTs. In the conventional meta-analysis (seven trials, 233 patients), the effect of CIMT on FIM was reported as significant (MD 2.88, 95% CI 0.08 to 5.68; P = 0.04). The diversity-adjusted required information size was 142 patients, and the cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.02 to 5.78). The effect of CIMT on ARAT (nine trials, 199 patients) was reported as significant (MD 7.78, 95% CI 1.19 to 14.37; P = 0.02). However, the diversity-adjusted required information size was 252 patients, and the Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).

Conclusions: Although conventional meta-analyses of CIMT reached statistical significance, their overall results remain inconclusive and might be spurious. Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

Show MeSH
Related in: MedlinePlus