Limits...
Power and sample size determination for the group comparison of patient-reported outcomes using the Rasch model: impact of a misspecification of the parameters.

Blanchin M, Guilleux A, Perrot B, Bonnaud-Antignac A, Hardouin JB, Sébille V - BMC Med Res Methodol (2015)

Bottom Line: The power of the test of the group effect estimated with Raschpower remains stable or shows a very little decrease whatever the values of the item parameters.A misspecification of the item difficulties regarding their overall pattern or their dispersion seems to have no or very little impact on the power of the test of the group effect.In contrast, a misspecification of the variance of the latent variable can have a strong impact as an underestimation of the variance will lead in some cases to an overestimation of the power at the design stage and may result in an underpowered study.

View Article: PubMed Central - PubMed

Affiliation: EA 4275, Biostatistics, Pharmacoepidemiology and Subjective Measures in Health Sciences, University of Nantes, 1 rue, Gaston Veil, 44000, Nantes, France. myriam.blanchin@univ-nantes.fr.

ABSTRACT

Background: Patient-reported outcomes (PRO) are important as endpoints in clinical trials and epidemiological studies. Guidelines for the development of PRO instruments and analysis of PRO data have emphasized the need to report methods used for sample size planning. The Raschpower procedure has been proposed for sample size and power determination for the comparison of PROs in cross-sectional studies comparing two groups of patients when an item reponse model, the Rasch model, is intended to be used for analysis. The power determination of the test of the group effect using Raschpower requires several parameters to be fixed at the planning stage including the item parameters and the variance of the latent variable. Wrong choices regarding these parameters can impact the expected power and the planned sample size to a greater or lesser extent depending on the magnitude of the erroneous assumptions.

Methods: The impact of a misspecification of the variance of the latent variable or of the item parameters on the determination of the power using the Raschpower procedure was investigated through the comparison of the estimations of the power in different situations.

Results: The power of the test of the group effect estimated with Raschpower remains stable or shows a very little decrease whatever the values of the item parameters. For most of the cases, the estimated power decreases when the variance of the latent trait increases. As a consequence, an underestimation of this variance will lead to an overestimation of the power of the group effect.

Conclusion: A misspecification of the item difficulties regarding their overall pattern or their dispersion seems to have no or very little impact on the power of the test of the group effect. In contrast, a misspecification of the variance of the latent variable can have a strong impact as an underestimation of the variance will lead in some cases to an overestimation of the power at the design stage and may result in an underpowered study.

No MeSH data available.


Density of mixture distribution for and different values ofa. Vertical lines represent the values of the item difficulties drawn from the mixture distribution. Item difficulties for a=−0.75 (Figure a): δj=(−1.13,−0.37,0.39,0.67,0.9). Item difficulties for a=0 (Figure b): δj=(−0.74,−0.29,0,0.29,0.74). Item difficulties for a=0.5 (Figure c): δj=(−0.81,−0.52,−0.26,0.13,1).
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4373307&req=5

Fig2: Density of mixture distribution for and different values ofa. Vertical lines represent the values of the item difficulties drawn from the mixture distribution. Item difficulties for a=−0.75 (Figure a): δj=(−1.13,−0.37,0.39,0.67,0.9). Item difficulties for a=0 (Figure b): δj=(−0.74,−0.29,0,0.29,0.74). Item difficulties for a=0.5 (Figure c): δj=(−0.81,−0.52,−0.26,0.13,1).

Mentions: The misspecification of the item difficulties was created using the variation of , 1.5,2,2.5,3,4,5,6,7,8,9} and of the gap between the means of the two normal distributions . A variation in the variance of the item distribution implies a variation of the intervals between the values of the item difficulties drawn from this distribution. An example of the effect of a variation of when J=5 and is represented in Figure 1. If the variance increases, e.g. from 1 in Figure 1(a) to 3 in Figure 1(b), the intervals between item difficulties increase. As a consequence, the easiest items become easier and the most difficult items become more difficult. An increase of the variance creates a shift of the item difficulties at both ends of the item difficulties distribution without changing the overall pattern. However, a variation of the gap between the means a leads to changing the overall pattern of the item difficulties as it alters the shape of the equiprobable mixture distribution as shown in Figure 2. We can note that for a=0 such as in Figure 2(b), a unimodal distribution is obtained and the item difficulties are almost regularly spaced. When a<0 such as in Figure 2(a), items difficulties on the left of the distribution are more spaced than item difficulties on the right and so the estimations of the latent variable will be more accurate on the right, and inversely when a>0 such as in Figure 2(c). Furthermore, to avoid ceiling and floor effects and ensure that the questionnaire was suitable for the population (not too specific nor too generic) [17], we decided to exclude cases where with , 7,8,9}. The other parameters used at the planning stage could also vary: the sample size in each group (N0=N1=50,100,200,300,500), the number of items (J=3,5,7,9,11,13,15), the value of the group effect (γ=0.1,0.2,0.5,0.8).Figure 1


Power and sample size determination for the group comparison of patient-reported outcomes using the Rasch model: impact of a misspecification of the parameters.

Blanchin M, Guilleux A, Perrot B, Bonnaud-Antignac A, Hardouin JB, Sébille V - BMC Med Res Methodol (2015)

Density of mixture distribution for and different values ofa. Vertical lines represent the values of the item difficulties drawn from the mixture distribution. Item difficulties for a=−0.75 (Figure a): δj=(−1.13,−0.37,0.39,0.67,0.9). Item difficulties for a=0 (Figure b): δj=(−0.74,−0.29,0,0.29,0.74). Item difficulties for a=0.5 (Figure c): δj=(−0.81,−0.52,−0.26,0.13,1).
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4373307&req=5

Fig2: Density of mixture distribution for and different values ofa. Vertical lines represent the values of the item difficulties drawn from the mixture distribution. Item difficulties for a=−0.75 (Figure a): δj=(−1.13,−0.37,0.39,0.67,0.9). Item difficulties for a=0 (Figure b): δj=(−0.74,−0.29,0,0.29,0.74). Item difficulties for a=0.5 (Figure c): δj=(−0.81,−0.52,−0.26,0.13,1).
Mentions: The misspecification of the item difficulties was created using the variation of , 1.5,2,2.5,3,4,5,6,7,8,9} and of the gap between the means of the two normal distributions . A variation in the variance of the item distribution implies a variation of the intervals between the values of the item difficulties drawn from this distribution. An example of the effect of a variation of when J=5 and is represented in Figure 1. If the variance increases, e.g. from 1 in Figure 1(a) to 3 in Figure 1(b), the intervals between item difficulties increase. As a consequence, the easiest items become easier and the most difficult items become more difficult. An increase of the variance creates a shift of the item difficulties at both ends of the item difficulties distribution without changing the overall pattern. However, a variation of the gap between the means a leads to changing the overall pattern of the item difficulties as it alters the shape of the equiprobable mixture distribution as shown in Figure 2. We can note that for a=0 such as in Figure 2(b), a unimodal distribution is obtained and the item difficulties are almost regularly spaced. When a<0 such as in Figure 2(a), items difficulties on the left of the distribution are more spaced than item difficulties on the right and so the estimations of the latent variable will be more accurate on the right, and inversely when a>0 such as in Figure 2(c). Furthermore, to avoid ceiling and floor effects and ensure that the questionnaire was suitable for the population (not too specific nor too generic) [17], we decided to exclude cases where with , 7,8,9}. The other parameters used at the planning stage could also vary: the sample size in each group (N0=N1=50,100,200,300,500), the number of items (J=3,5,7,9,11,13,15), the value of the group effect (γ=0.1,0.2,0.5,0.8).Figure 1

Bottom Line: The power of the test of the group effect estimated with Raschpower remains stable or shows a very little decrease whatever the values of the item parameters.A misspecification of the item difficulties regarding their overall pattern or their dispersion seems to have no or very little impact on the power of the test of the group effect.In contrast, a misspecification of the variance of the latent variable can have a strong impact as an underestimation of the variance will lead in some cases to an overestimation of the power at the design stage and may result in an underpowered study.

View Article: PubMed Central - PubMed

Affiliation: EA 4275, Biostatistics, Pharmacoepidemiology and Subjective Measures in Health Sciences, University of Nantes, 1 rue, Gaston Veil, 44000, Nantes, France. myriam.blanchin@univ-nantes.fr.

ABSTRACT

Background: Patient-reported outcomes (PRO) are important as endpoints in clinical trials and epidemiological studies. Guidelines for the development of PRO instruments and analysis of PRO data have emphasized the need to report methods used for sample size planning. The Raschpower procedure has been proposed for sample size and power determination for the comparison of PROs in cross-sectional studies comparing two groups of patients when an item reponse model, the Rasch model, is intended to be used for analysis. The power determination of the test of the group effect using Raschpower requires several parameters to be fixed at the planning stage including the item parameters and the variance of the latent variable. Wrong choices regarding these parameters can impact the expected power and the planned sample size to a greater or lesser extent depending on the magnitude of the erroneous assumptions.

Methods: The impact of a misspecification of the variance of the latent variable or of the item parameters on the determination of the power using the Raschpower procedure was investigated through the comparison of the estimations of the power in different situations.

Results: The power of the test of the group effect estimated with Raschpower remains stable or shows a very little decrease whatever the values of the item parameters. For most of the cases, the estimated power decreases when the variance of the latent trait increases. As a consequence, an underestimation of this variance will lead to an overestimation of the power of the group effect.

Conclusion: A misspecification of the item difficulties regarding their overall pattern or their dispersion seems to have no or very little impact on the power of the test of the group effect. In contrast, a misspecification of the variance of the latent variable can have a strong impact as an underestimation of the variance will lead in some cases to an overestimation of the power at the design stage and may result in an underpowered study.

No MeSH data available.