Limits...
The need for randomization in animal trials: an overview of systematic reviews.

Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, Heneghan C - PLoS ONE (2014)

Bottom Line: We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not.In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom.

ABSTRACT

Background and objectives: Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition.

Methods: We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment.

Results: Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinson's disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.

Conclusions: Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Show MeSH

Related in: MedlinePlus

Forest plot showing the effect of allocation concealment on effect size.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4048216&req=5

pone-0098856-g003: Forest plot showing the effect of allocation concealment on effect size.

Mentions: Eighteen reviews with 2696 comparisons (39,405 animals) reported the effect of allocation concealment. Studies in which allocation concealment was used resulted in slightly decreased effect sizes, but this was not statistically significant (SMD  =  −0.04, 95% CI −0.09 to 0.00, I2 = 51.6%, P = 0.059) (Figure 3). Subgroup analysis examining different diseases (stroke and non-stroke) showed that allocation concealment in studies of stroke resulted in significantly lower effect sizes (SMD =  −0.07, 95% CI −0.12 to −0.02, I2 = 48.5%, P = 0.009), whereas allocation concealment in other disease areas resulted in higher effect sizes (SMD 0.05, 95% CI −0.01 to 0.11, I2 = 0%, P = 0.128) but the difference between these groups was not found to be significant using meta-regression (P = 0.073). Meta-regression of the combination of disease and outcome measure was did not explain more than 9% of the heterogeneity. In our secondary analysis (where we ignored direction of effect) we found a larger difference between concealed and non-concealed studies (SMD −0.08, 95% CI −0.11 to −0.05, I2 = 13.8%, P<0.0001) compared with the effect size in which we took direction into consideration.


The need for randomization in animal trials: an overview of systematic reviews.

Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, Heneghan C - PLoS ONE (2014)

Forest plot showing the effect of allocation concealment on effect size.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4048216&req=5

pone-0098856-g003: Forest plot showing the effect of allocation concealment on effect size.
Mentions: Eighteen reviews with 2696 comparisons (39,405 animals) reported the effect of allocation concealment. Studies in which allocation concealment was used resulted in slightly decreased effect sizes, but this was not statistically significant (SMD  =  −0.04, 95% CI −0.09 to 0.00, I2 = 51.6%, P = 0.059) (Figure 3). Subgroup analysis examining different diseases (stroke and non-stroke) showed that allocation concealment in studies of stroke resulted in significantly lower effect sizes (SMD =  −0.07, 95% CI −0.12 to −0.02, I2 = 48.5%, P = 0.009), whereas allocation concealment in other disease areas resulted in higher effect sizes (SMD 0.05, 95% CI −0.01 to 0.11, I2 = 0%, P = 0.128) but the difference between these groups was not found to be significant using meta-regression (P = 0.073). Meta-regression of the combination of disease and outcome measure was did not explain more than 9% of the heterogeneity. In our secondary analysis (where we ignored direction of effect) we found a larger difference between concealed and non-concealed studies (SMD −0.08, 95% CI −0.11 to −0.05, I2 = 13.8%, P<0.0001) compared with the effect size in which we took direction into consideration.

Bottom Line: We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not.In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom.

ABSTRACT

Background and objectives: Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition.

Methods: We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment.

Results: Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinson's disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.

Conclusions: Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Show MeSH
Related in: MedlinePlus