Limits...
Interpreting the results of a retrospective comparison of test and reference treatments in a randomized clinical trial setting.

Fridman M, Erder MH - Clin Drug Investig (2015)

Bottom Line: Retrospective analyses showed that the improvement in the symptoms of ADHD was greater in patients treated with LDX than OROS-MPH.We now show that this observation remained significant after the application of the four statistical penalties.By adjusting the significance level, it is possible to compare quantitatively such retrospective results with prospectively defined comparisons.

View Article: PubMed Central - PubMed

Affiliation: AMF Consulting, Inc., Los Angeles, CA, USA, fmoshe@amf-consulting.com.

ABSTRACT

Background and objectives: The retrospective comparison of test and reference treatment arms in a randomized prospective clinical trial is potentially useful in economic modeling seeking to assess the cost effectiveness of alternative therapies.

Methods: To enhance the credibility of such retrospective comparisons, we propose the application of the following adjustments to significance levels obtained from standard statistical methodology: (1) a significance test for the lower bound of the 95 % confidence interval for the observed difference, (2) a conservative Bonferroni method of adjustment for multiple comparisons, (3) an adjusted p-value calculated using Scheffe's single-step method, and (4) Bayesian 95 % credibility intervals with a prior centered at zero.

Results: These adjustments were applied to data from a randomized double-blind concurrent trial (SPD489-325) that established the efficacy and safety of lisdexamfetamine dimesylate (LDX) in children and adolescents with attention-deficit/hyperactivity disorder (ADHD). Prospectively planned analyses demonstrated that the reduction in the symptoms of ADHD was significantly greater than placebo in patients treated with either LDX or the reference treatment, osmotic-release oral system methylphenidate (OROS-MPH). Retrospective analyses showed that the improvement in the symptoms of ADHD was greater in patients treated with LDX than OROS-MPH. We now show that this observation remained significant after the application of the four statistical penalties.

Conclusions: By adjusting the significance level, it is possible to compare quantitatively such retrospective results with prospectively defined comparisons. However, the qualitative level of such retrospective evidence should remain secondary to that obtained from prospectively specified comparisons in a randomized clinical trial.

Show MeSH

Related in: MedlinePlus

Sensitivity analysis of the 95 % Bayesian credibility intervals (Method 4) over mean and standard deviation of the prior distribution. CI credibility interval, SD standard deviation
© Copyright Policy - OpenAccess
Related In: Results  -  Collection


getmorefigures.php?uid=PMC4300432&req=5

Fig1: Sensitivity analysis of the 95 % Bayesian credibility intervals (Method 4) over mean and standard deviation of the prior distribution. CI credibility interval, SD standard deviation

Mentions: For Method 4, the degree of penalty can vary depending on the prior distribution used. Clearly, the more distant towards the opposite side of zero the prior mean is relative to the observed mean difference between treatments, the smaller the prior variance value and the larger the sample size for the source of the prior information, the stronger the prior evidence against the observed results and the stricter the penalty. Figure 1 presents a sensitivity analysis of the credibility intervals as the mean and standard deviation of the assumed prior distribution are varied.Fig. 1


Interpreting the results of a retrospective comparison of test and reference treatments in a randomized clinical trial setting.

Fridman M, Erder MH - Clin Drug Investig (2015)

Sensitivity analysis of the 95 % Bayesian credibility intervals (Method 4) over mean and standard deviation of the prior distribution. CI credibility interval, SD standard deviation
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC4300432&req=5

Fig1: Sensitivity analysis of the 95 % Bayesian credibility intervals (Method 4) over mean and standard deviation of the prior distribution. CI credibility interval, SD standard deviation
Mentions: For Method 4, the degree of penalty can vary depending on the prior distribution used. Clearly, the more distant towards the opposite side of zero the prior mean is relative to the observed mean difference between treatments, the smaller the prior variance value and the larger the sample size for the source of the prior information, the stronger the prior evidence against the observed results and the stricter the penalty. Figure 1 presents a sensitivity analysis of the credibility intervals as the mean and standard deviation of the assumed prior distribution are varied.Fig. 1

Bottom Line: Retrospective analyses showed that the improvement in the symptoms of ADHD was greater in patients treated with LDX than OROS-MPH.We now show that this observation remained significant after the application of the four statistical penalties.By adjusting the significance level, it is possible to compare quantitatively such retrospective results with prospectively defined comparisons.

View Article: PubMed Central - PubMed

Affiliation: AMF Consulting, Inc., Los Angeles, CA, USA, fmoshe@amf-consulting.com.

ABSTRACT

Background and objectives: The retrospective comparison of test and reference treatment arms in a randomized prospective clinical trial is potentially useful in economic modeling seeking to assess the cost effectiveness of alternative therapies.

Methods: To enhance the credibility of such retrospective comparisons, we propose the application of the following adjustments to significance levels obtained from standard statistical methodology: (1) a significance test for the lower bound of the 95 % confidence interval for the observed difference, (2) a conservative Bonferroni method of adjustment for multiple comparisons, (3) an adjusted p-value calculated using Scheffe's single-step method, and (4) Bayesian 95 % credibility intervals with a prior centered at zero.

Results: These adjustments were applied to data from a randomized double-blind concurrent trial (SPD489-325) that established the efficacy and safety of lisdexamfetamine dimesylate (LDX) in children and adolescents with attention-deficit/hyperactivity disorder (ADHD). Prospectively planned analyses demonstrated that the reduction in the symptoms of ADHD was significantly greater than placebo in patients treated with either LDX or the reference treatment, osmotic-release oral system methylphenidate (OROS-MPH). Retrospective analyses showed that the improvement in the symptoms of ADHD was greater in patients treated with LDX than OROS-MPH. We now show that this observation remained significant after the application of the four statistical penalties.

Conclusions: By adjusting the significance level, it is possible to compare quantitatively such retrospective results with prospectively defined comparisons. However, the qualitative level of such retrospective evidence should remain secondary to that obtained from prospectively specified comparisons in a randomized clinical trial.

Show MeSH
Related in: MedlinePlus