Limits...
Item response theory-based measure of global disability in multiple sclerosis derived from the Performance Scales and related items.

Chamot E, Kister I, Cutter GR - BMC Neurol (2014)

Bottom Line: The construct validity of the three scores was compared by examining the magnitude of their associations with participant's background characteristics, including unemployment.We derived structurally valid measures of global disability from the PS through the proposed methodology that were superior to the PSS.Higher scores of global disability were significantly associated with older age at assessment, longer disease duration, male gender, Native-American ethnicity, not receiving disease modifying therapy, unemployment, and higher scores on the Patient Determined Disease Steps (PDDS).

View Article: PubMed Central - PubMed

Affiliation: Department of Epidemiology, University of Alabama at Birmingham School of Public Health, 1665 University Blvd, Suite 217H, Birmingham 35294-0022, AL, USA. echamot@uab.edu.

ABSTRACT

Background: The eight Performance Scales and three assimilated scales (PS) used in North American Research Committee on Multiple Sclerosis (NARCOMS) registry surveys cover a broad range of neurologic domains commonly affected by multiple sclerosis (mobility, hand function, vision, fatigue, cognition, bladder/bowel, sensory, spasticity, pain, depression, and tremor/coordination). Each scale consists of a single 6-to-7-point Likert item with response categories ranging from "normal" to "total disability". Relatively little is known about the performances of the summary index of disability derived from these scales (the Performance Scales Sum or PSS). In this study, we demonstrate the value of a combination of classical and modern methods recently proposed by the Patient-Reported Outcome Measurement Information System (PROMIS) network to evaluate the psychometric properties of the PSS and derive an improved measure of global disability from the PS.

Methods: The study sample included 7,851adults with MS who completed a NARCOMS intake questionnaire between 2003 and 2011. Factor analysis, bifactor modeling, and item response theory (IRT) analysis were used to evaluate the dimension(s) of disability underlying the PS; calibrate the 11 scales; and generate three alternative summary scores of global disability corresponding to different model assumptions and practical priorities. The construct validity of the three scores was compared by examining the magnitude of their associations with participant's background characteristics, including unemployment.

Results: We derived structurally valid measures of global disability from the PS through the proposed methodology that were superior to the PSS. The measure most applicable to clinical practice gives similar weight to physical and mental disability. Overall reliability of the new measure is acceptable for individual comparisons (0.87). Higher scores of global disability were significantly associated with older age at assessment, longer disease duration, male gender, Native-American ethnicity, not receiving disease modifying therapy, unemployment, and higher scores on the Patient Determined Disease Steps (PDDS).

Conclusion: Promising, interpretable and easily-obtainable IRT scores of global disability were generated from the PS by using a sequence of traditional and modern psychometric methods based on PROMIS recommendations. Our analyses shed new light on the construct of global disability in MS.

No MeSH data available.


Related in: MedlinePlus

Bifactor CFA model of self-assessed neurological disability in NARCOMS registrants. Note: “Global” represents the general factor of global disability; “Physical” and “Mental” represent the auxiliary factors of “physical” and “mental” disability. Correlations among the three factors are all forced to be zero. Thus, the physical and mental factors each explain a fraction of the variability in PS scores left unexplained by the general factor. Comparisons of “Residual variances” in Figure 2 and Figure 1, provide information about the fraction of variability in PS scores that the two auxiliary factors explain above and beyond the general factor.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4195863&req=5

Fig2: Bifactor CFA model of self-assessed neurological disability in NARCOMS registrants. Note: “Global” represents the general factor of global disability; “Physical” and “Mental” represent the auxiliary factors of “physical” and “mental” disability. Correlations among the three factors are all forced to be zero. Thus, the physical and mental factors each explain a fraction of the variability in PS scores left unexplained by the general factor. Comparisons of “Residual variances” in Figure 2 and Figure 1, provide information about the fraction of variability in PS scores that the two auxiliary factors explain above and beyond the general factor.

Mentions: The bifactor CFA model was specified so that the sensory PS contributed only to the general factor (i.e., to what the physical and mental PS measured in common; Figure 2). The fit of this model was excellent. Correlations between the PS and the factor of global disability were very similar to their counterpart in the unidimensional model. The largest differences in factor-PS correlations were observed for the mobility PS (correlation of 0.55 in the bifactor model vs. 0.66 in the unidimensional model) and the cognition PS (correlation of 0.56 in the bifactor model vs. 0.65 in the unidimensional model) . Both differences matched the accepted standard of ≤0.15 for a small difference [24]. This suggested that the mobility and cognitive PS would be only slightly overrepresented in scores of global disability obtained from the parsimonious, unidimensional, IRT model compared to scores of global disability obtained from the more complex bifactor IRT model. Furthermore, the variance of PS sum scores was decomposed into a large fraction explained by the factor of global disability (79%), a small fraction explained by the two auxiliary factors (11%), and a small fraction of residual error (10%) [25]. Expressed differently, 87.8% of reliable variance in the sum score represented global disability as opposed to domain-specific disability. This result was in line with the finding that only two PS had salient correlations with the factor of residual physical disability (mobility, 0.81 and tremor/coordination, 0.33) and three with the factor of residual mental disability (cognition, 0.56; depression, 0.39; and vision, 0.32). Since the auxiliary factors of a bifactor model are considered to be minor and ill-defined if they include less than three items with item-factor correlations ≥ 0.40-0.50 [26], we concluded that scores of residual physical and mental disability might not be estimated with sufficient accuracy to be of practical importance in less than very large studies.Figure 2


Item response theory-based measure of global disability in multiple sclerosis derived from the Performance Scales and related items.

Chamot E, Kister I, Cutter GR - BMC Neurol (2014)

Bifactor CFA model of self-assessed neurological disability in NARCOMS registrants. Note: “Global” represents the general factor of global disability; “Physical” and “Mental” represent the auxiliary factors of “physical” and “mental” disability. Correlations among the three factors are all forced to be zero. Thus, the physical and mental factors each explain a fraction of the variability in PS scores left unexplained by the general factor. Comparisons of “Residual variances” in Figure 2 and Figure 1, provide information about the fraction of variability in PS scores that the two auxiliary factors explain above and beyond the general factor.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4195863&req=5

Fig2: Bifactor CFA model of self-assessed neurological disability in NARCOMS registrants. Note: “Global” represents the general factor of global disability; “Physical” and “Mental” represent the auxiliary factors of “physical” and “mental” disability. Correlations among the three factors are all forced to be zero. Thus, the physical and mental factors each explain a fraction of the variability in PS scores left unexplained by the general factor. Comparisons of “Residual variances” in Figure 2 and Figure 1, provide information about the fraction of variability in PS scores that the two auxiliary factors explain above and beyond the general factor.
Mentions: The bifactor CFA model was specified so that the sensory PS contributed only to the general factor (i.e., to what the physical and mental PS measured in common; Figure 2). The fit of this model was excellent. Correlations between the PS and the factor of global disability were very similar to their counterpart in the unidimensional model. The largest differences in factor-PS correlations were observed for the mobility PS (correlation of 0.55 in the bifactor model vs. 0.66 in the unidimensional model) and the cognition PS (correlation of 0.56 in the bifactor model vs. 0.65 in the unidimensional model) . Both differences matched the accepted standard of ≤0.15 for a small difference [24]. This suggested that the mobility and cognitive PS would be only slightly overrepresented in scores of global disability obtained from the parsimonious, unidimensional, IRT model compared to scores of global disability obtained from the more complex bifactor IRT model. Furthermore, the variance of PS sum scores was decomposed into a large fraction explained by the factor of global disability (79%), a small fraction explained by the two auxiliary factors (11%), and a small fraction of residual error (10%) [25]. Expressed differently, 87.8% of reliable variance in the sum score represented global disability as opposed to domain-specific disability. This result was in line with the finding that only two PS had salient correlations with the factor of residual physical disability (mobility, 0.81 and tremor/coordination, 0.33) and three with the factor of residual mental disability (cognition, 0.56; depression, 0.39; and vision, 0.32). Since the auxiliary factors of a bifactor model are considered to be minor and ill-defined if they include less than three items with item-factor correlations ≥ 0.40-0.50 [26], we concluded that scores of residual physical and mental disability might not be estimated with sufficient accuracy to be of practical importance in less than very large studies.Figure 2

Bottom Line: The construct validity of the three scores was compared by examining the magnitude of their associations with participant's background characteristics, including unemployment.We derived structurally valid measures of global disability from the PS through the proposed methodology that were superior to the PSS.Higher scores of global disability were significantly associated with older age at assessment, longer disease duration, male gender, Native-American ethnicity, not receiving disease modifying therapy, unemployment, and higher scores on the Patient Determined Disease Steps (PDDS).

View Article: PubMed Central - PubMed

Affiliation: Department of Epidemiology, University of Alabama at Birmingham School of Public Health, 1665 University Blvd, Suite 217H, Birmingham 35294-0022, AL, USA. echamot@uab.edu.

ABSTRACT

Background: The eight Performance Scales and three assimilated scales (PS) used in North American Research Committee on Multiple Sclerosis (NARCOMS) registry surveys cover a broad range of neurologic domains commonly affected by multiple sclerosis (mobility, hand function, vision, fatigue, cognition, bladder/bowel, sensory, spasticity, pain, depression, and tremor/coordination). Each scale consists of a single 6-to-7-point Likert item with response categories ranging from "normal" to "total disability". Relatively little is known about the performances of the summary index of disability derived from these scales (the Performance Scales Sum or PSS). In this study, we demonstrate the value of a combination of classical and modern methods recently proposed by the Patient-Reported Outcome Measurement Information System (PROMIS) network to evaluate the psychometric properties of the PSS and derive an improved measure of global disability from the PS.

Methods: The study sample included 7,851adults with MS who completed a NARCOMS intake questionnaire between 2003 and 2011. Factor analysis, bifactor modeling, and item response theory (IRT) analysis were used to evaluate the dimension(s) of disability underlying the PS; calibrate the 11 scales; and generate three alternative summary scores of global disability corresponding to different model assumptions and practical priorities. The construct validity of the three scores was compared by examining the magnitude of their associations with participant's background characteristics, including unemployment.

Results: We derived structurally valid measures of global disability from the PS through the proposed methodology that were superior to the PSS. The measure most applicable to clinical practice gives similar weight to physical and mental disability. Overall reliability of the new measure is acceptable for individual comparisons (0.87). Higher scores of global disability were significantly associated with older age at assessment, longer disease duration, male gender, Native-American ethnicity, not receiving disease modifying therapy, unemployment, and higher scores on the Patient Determined Disease Steps (PDDS).

Conclusion: Promising, interpretable and easily-obtainable IRT scores of global disability were generated from the PS by using a sequence of traditional and modern psychometric methods based on PROMIS recommendations. Our analyses shed new light on the construct of global disability in MS.

No MeSH data available.


Related in: MedlinePlus