Limits...
The need for randomization in animal trials: an overview of systematic reviews.

Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, Heneghan C - PLoS ONE (2014)

Bottom Line: We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not.In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom.

ABSTRACT

Background and objectives: Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition.

Methods: We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment.

Results: Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinson's disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.

Conclusions: Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Show MeSH

Related in: MedlinePlus

Forest plot showing the effect of random allocation on effect size.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4048216&req=5

pone-0098856-g002: Forest plot showing the effect of random allocation on effect size.

Mentions: Thirty reviews with 7249 comparisons (121,784 animals) reported the effects of randomization. Randomized trials reduced effect sizes by a moderate and statistically significant amount (SMD  =  −0.07, 95% CI −0.12 to −0.02, I2 = 89.1%, P  =  0.008) (Figure 2). In a subgroup analysis examining the effect of randomization by disease (stroke versus other), we found that randomization resulted in a lower effect size in areas other than stroke (SMD −0.18, 95% CI −0.30 to −0.06) but not stroke itself (SMD −0.03 95% CI −0.08 to 0.02). However, using meta-regression we found no significant difference between stroke and non-stroke on outcome measures (P  = 0.08); additionally, meta-regression could not explain more than 3% of the heterogeneity. A sensitivity analysis excluding the single review [32] in which we had to estimate the number of animals, did not alter the overall result (SMD =  −0.08 95% CI −0.13 to −0.03). In our secondary analysis (where we ignored direction of effect) we found a larger difference between randomized and non-randomized studies (SMD −0.16, 95% CI −0.21 to −0.11, I2 = 86.6%, P<0.0001) compared with the effect size in which we took direction into consideration.


The need for randomization in animal trials: an overview of systematic reviews.

Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, Heneghan C - PLoS ONE (2014)

Forest plot showing the effect of random allocation on effect size.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4048216&req=5

pone-0098856-g002: Forest plot showing the effect of random allocation on effect size.
Mentions: Thirty reviews with 7249 comparisons (121,784 animals) reported the effects of randomization. Randomized trials reduced effect sizes by a moderate and statistically significant amount (SMD  =  −0.07, 95% CI −0.12 to −0.02, I2 = 89.1%, P  =  0.008) (Figure 2). In a subgroup analysis examining the effect of randomization by disease (stroke versus other), we found that randomization resulted in a lower effect size in areas other than stroke (SMD −0.18, 95% CI −0.30 to −0.06) but not stroke itself (SMD −0.03 95% CI −0.08 to 0.02). However, using meta-regression we found no significant difference between stroke and non-stroke on outcome measures (P  = 0.08); additionally, meta-regression could not explain more than 3% of the heterogeneity. A sensitivity analysis excluding the single review [32] in which we had to estimate the number of animals, did not alter the overall result (SMD =  −0.08 95% CI −0.13 to −0.03). In our secondary analysis (where we ignored direction of effect) we found a larger difference between randomized and non-randomized studies (SMD −0.16, 95% CI −0.21 to −0.11, I2 = 86.6%, P<0.0001) compared with the effect size in which we took direction into consideration.

Bottom Line: We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not.In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom.

ABSTRACT

Background and objectives: Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition.

Methods: We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment.

Results: Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinson's disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.

Conclusions: Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Show MeSH
Related in: MedlinePlus