Limits...
Assessing potential sources of clustering in individually randomised trials.

Kahan BC, Morris TP - BMC Med Res Methodol (2013)

Bottom Line: Recent reviews have shown that while clustering is extremely common in individually randomised trials (for example, clustering within centre, therapist, or surgeon), it is rarely accounted for in the trial analysis.A simulation study is used to explore the impact of not accounting for non-ignorable clustering in practice.Clustering is non-ignorable when there is both correlation between patient outcomes within clusters, and correlation between treatment assignments within clusters.

View Article: PubMed Central - HTML - PubMed

Affiliation: MRC Clinical Trials Unit, Aviation House, 125 Kingsway, London, WC2B 6NH, UK. brk@ctu.mrc.ac.uk

ABSTRACT

Background: Recent reviews have shown that while clustering is extremely common in individually randomised trials (for example, clustering within centre, therapist, or surgeon), it is rarely accounted for in the trial analysis. Our aim is to develop a general framework for assessing whether potential sources of clustering must be accounted for in the trial analysis to obtain valid type I error rates (non-ignorable clustering), with a particular focus on individually randomised trials.

Methods: A general framework for assessing clustering is developed based on theoretical results and a case study of a recently published trial is used to illustrate the concepts. A simulation study is used to explore the impact of not accounting for non-ignorable clustering in practice.

Results: Clustering is non-ignorable when there is both correlation between patient outcomes within clusters, and correlation between treatment assignments within clusters. This occurs when the intraclass correlation coefficient is non-zero, and when the cluster has been used in the randomisation process (e.g. stratified blocks within centre) or when patients are assigned to clusters after randomisation with different probabilities (e.g. a surgery trial in which surgeons treat patients in only one arm). A case study of an individually randomised trial found multiple sources of clustering, including centre of recruitment, attending surgeon, and site of rehabilitation class. Simulations show that failure to account for non-ignorable clustering in trial analyses can lead to type I error rates over 20% in certain cases; conversely, adjusting for the clustering in the trial analysis gave correct type I error rates.

Conclusions: Clustering is common in individually randomised trials. Trialists should assess potential sources of clustering during the planning stages of a trial, and account for any sources of non-ignorable clustering in the trial analysis.

Show MeSH
Loss in power from not accounting for ignorable clustering in the analysis.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3643875&req=5

Figure 5: Loss in power from not accounting for ignorable clustering in the analysis.

Mentions: When clustering is ignorable, unadjusted analysis will still give valid type I error rates. However, if the ICC is high, then an unadjusted analysis will lead to a loss of power, making it more difficult to detect a treatment effect. For example, in a therapy trial, if therapists have a large effect on the outcome, but are not any more likely to treat patients from a specific treatment group, then therapists are ignorable, and an analysis that does not account for therapist effects will still give valid results. However, adjusting for therapist in the analysis may be preferable as it will increase power. This is demonstrated in FigureĀ 5, which shows the expected loss in power for trials which do not account for ignorable clustering in the trial (given the trial is powered at 80%). For large ICCs, the loss in power can be substantial; for example, ICCs of 0.10, 0.20, and 0.30 would lead to a reduction in power of 4%, 9%, and 15% respectively. Further work on the benefits of adjusting for factors associated with the outcome can be found elsewhere [16,17].


Assessing potential sources of clustering in individually randomised trials.

Kahan BC, Morris TP - BMC Med Res Methodol (2013)

Loss in power from not accounting for ignorable clustering in the analysis.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3643875&req=5

Figure 5: Loss in power from not accounting for ignorable clustering in the analysis.
Mentions: When clustering is ignorable, unadjusted analysis will still give valid type I error rates. However, if the ICC is high, then an unadjusted analysis will lead to a loss of power, making it more difficult to detect a treatment effect. For example, in a therapy trial, if therapists have a large effect on the outcome, but are not any more likely to treat patients from a specific treatment group, then therapists are ignorable, and an analysis that does not account for therapist effects will still give valid results. However, adjusting for therapist in the analysis may be preferable as it will increase power. This is demonstrated in FigureĀ 5, which shows the expected loss in power for trials which do not account for ignorable clustering in the trial (given the trial is powered at 80%). For large ICCs, the loss in power can be substantial; for example, ICCs of 0.10, 0.20, and 0.30 would lead to a reduction in power of 4%, 9%, and 15% respectively. Further work on the benefits of adjusting for factors associated with the outcome can be found elsewhere [16,17].

Bottom Line: Recent reviews have shown that while clustering is extremely common in individually randomised trials (for example, clustering within centre, therapist, or surgeon), it is rarely accounted for in the trial analysis.A simulation study is used to explore the impact of not accounting for non-ignorable clustering in practice.Clustering is non-ignorable when there is both correlation between patient outcomes within clusters, and correlation between treatment assignments within clusters.

View Article: PubMed Central - HTML - PubMed

Affiliation: MRC Clinical Trials Unit, Aviation House, 125 Kingsway, London, WC2B 6NH, UK. brk@ctu.mrc.ac.uk

ABSTRACT

Background: Recent reviews have shown that while clustering is extremely common in individually randomised trials (for example, clustering within centre, therapist, or surgeon), it is rarely accounted for in the trial analysis. Our aim is to develop a general framework for assessing whether potential sources of clustering must be accounted for in the trial analysis to obtain valid type I error rates (non-ignorable clustering), with a particular focus on individually randomised trials.

Methods: A general framework for assessing clustering is developed based on theoretical results and a case study of a recently published trial is used to illustrate the concepts. A simulation study is used to explore the impact of not accounting for non-ignorable clustering in practice.

Results: Clustering is non-ignorable when there is both correlation between patient outcomes within clusters, and correlation between treatment assignments within clusters. This occurs when the intraclass correlation coefficient is non-zero, and when the cluster has been used in the randomisation process (e.g. stratified blocks within centre) or when patients are assigned to clusters after randomisation with different probabilities (e.g. a surgery trial in which surgeons treat patients in only one arm). A case study of an individually randomised trial found multiple sources of clustering, including centre of recruitment, attending surgeon, and site of rehabilitation class. Simulations show that failure to account for non-ignorable clustering in trial analyses can lead to type I error rates over 20% in certain cases; conversely, adjusting for the clustering in the trial analysis gave correct type I error rates.

Conclusions: Clustering is common in individually randomised trials. Trialists should assess potential sources of clustering during the planning stages of a trial, and account for any sources of non-ignorable clustering in the trial analysis.

Show MeSH