Limits...
Questionnaire discrimination: (re)-introducing coefficient delta.

Hankins M - BMC Med Res Methodol (2007)

Bottom Line: However, deltaG was substantially lower (0.73) for the dichotomous coding of the GHQ-12 than for the Likert-type method (deltaG = 0.96), indicating that the dichotomous coding, although reliable, failed to discriminate between individuals.Ferguson's delta has been neglected in discussions of questionnaire design and performance, perhaps because it has not been implemented in software and was restricted to questionnaires with dichotomous items, which are rare in health care research.It is suggested that the more general formula introduced here is reported as deltaG, to avoid the implication that items are dichotomously coded.

View Article: PubMed Central - HTML - PubMed

Affiliation: King's College London, Department of Psychology (at Guy's), Institute of Psychiatry, London, UK. matthew.hankins@kcl.ac.uk

ABSTRACT

Background: Questionnaires are used routinely in clinical research to measure health status and quality of life. Questionnaire measurements are traditionally formally assessed by indices of reliability (the degree of measurement error) and validity (the extent to which the questionnaire measures what it is supposed to measure). Neither of these indices assesses the degree to which the questionnaire is able to discriminate between individuals, an important aspect of measurement. This paper introduces and extends an existing index of a questionnaire's ability to distinguish between individuals, that is, the questionnaire's discrimination.

Methods: Ferguson (1949) 1 derived an index of test discrimination, coefficient delta, for psychometric tests with dichotomous (correct/incorrect) items. In this paper a general form of the formula, deltaG, is derived for the more general class of questionnaires allowing for several response choices. The calculation and characteristics of deltaG are then demonstrated using questionnaire data (GHQ-12) from 2003-2004 British Household Panel Survey (N = 14761). Coefficients for reliability (alpha) and discrimination (deltaG) are computed for two commonly-used GHQ-12 coding methods: dichotomous coding and four-point Likert-type coding.

Results: Both scoring methods were reliable (alpha > 0.88). However, deltaG was substantially lower (0.73) for the dichotomous coding of the GHQ-12 than for the Likert-type method (deltaG = 0.96), indicating that the dichotomous coding, although reliable, failed to discriminate between individuals.

Conclusion: Coefficient deltaG was shown to have decisive utility in distinguishing between the cross-sectional discrimination of two equally reliable scoring methods. Ferguson's delta has been neglected in discussions of questionnaire design and performance, perhaps because it has not been implemented in software and was restricted to questionnaires with dichotomous items, which are rare in health care research. It is suggested that the more general formula introduced here is reported as deltaG, to avoid the implication that items are dichotomously coded.

Show MeSH

Related in: MedlinePlus

Frequency histograms and boxplots for GHQ-12 dichotomous and polytomous codings (N = 14761).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC1884165&req=5

Figure 1: Frequency histograms and boxplots for GHQ-12 dichotomous and polytomous codings (N = 14761).

Mentions: As can be seen in Figure 1, the distribution of the GHQ-12 score was greatly affected by the scoring method. Polytomous coding produced a slightly skewed distribution but one with clearly defined tails (skew = 1.3, SE = 0.02), with discrimination δG = 0.96 (actual value: 0.957; bootstrapped 95% CL: 0.956, 0.959) and reliability of α = 0.88. Dichotomous coding resulted in a highly-skewed distribution (skew = 1.86, SE = 0.02) with 54.2% of the sample scoring the scale minimum: this lack of discrimination was reflected in the value of δG = 0.73 (actual value: 0.731; bootstrapped 95% CL: 0.723, 0.739). Reliability was α = 0.89. The two scoring methods were highly correlated (r = 0.90, p < 0.001).


Questionnaire discrimination: (re)-introducing coefficient delta.

Hankins M - BMC Med Res Methodol (2007)

Frequency histograms and boxplots for GHQ-12 dichotomous and polytomous codings (N = 14761).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC1884165&req=5

Figure 1: Frequency histograms and boxplots for GHQ-12 dichotomous and polytomous codings (N = 14761).
Mentions: As can be seen in Figure 1, the distribution of the GHQ-12 score was greatly affected by the scoring method. Polytomous coding produced a slightly skewed distribution but one with clearly defined tails (skew = 1.3, SE = 0.02), with discrimination δG = 0.96 (actual value: 0.957; bootstrapped 95% CL: 0.956, 0.959) and reliability of α = 0.88. Dichotomous coding resulted in a highly-skewed distribution (skew = 1.86, SE = 0.02) with 54.2% of the sample scoring the scale minimum: this lack of discrimination was reflected in the value of δG = 0.73 (actual value: 0.731; bootstrapped 95% CL: 0.723, 0.739). Reliability was α = 0.89. The two scoring methods were highly correlated (r = 0.90, p < 0.001).

Bottom Line: However, deltaG was substantially lower (0.73) for the dichotomous coding of the GHQ-12 than for the Likert-type method (deltaG = 0.96), indicating that the dichotomous coding, although reliable, failed to discriminate between individuals.Ferguson's delta has been neglected in discussions of questionnaire design and performance, perhaps because it has not been implemented in software and was restricted to questionnaires with dichotomous items, which are rare in health care research.It is suggested that the more general formula introduced here is reported as deltaG, to avoid the implication that items are dichotomously coded.

View Article: PubMed Central - HTML - PubMed

Affiliation: King's College London, Department of Psychology (at Guy's), Institute of Psychiatry, London, UK. matthew.hankins@kcl.ac.uk

ABSTRACT

Background: Questionnaires are used routinely in clinical research to measure health status and quality of life. Questionnaire measurements are traditionally formally assessed by indices of reliability (the degree of measurement error) and validity (the extent to which the questionnaire measures what it is supposed to measure). Neither of these indices assesses the degree to which the questionnaire is able to discriminate between individuals, an important aspect of measurement. This paper introduces and extends an existing index of a questionnaire's ability to distinguish between individuals, that is, the questionnaire's discrimination.

Methods: Ferguson (1949) 1 derived an index of test discrimination, coefficient delta, for psychometric tests with dichotomous (correct/incorrect) items. In this paper a general form of the formula, deltaG, is derived for the more general class of questionnaires allowing for several response choices. The calculation and characteristics of deltaG are then demonstrated using questionnaire data (GHQ-12) from 2003-2004 British Household Panel Survey (N = 14761). Coefficients for reliability (alpha) and discrimination (deltaG) are computed for two commonly-used GHQ-12 coding methods: dichotomous coding and four-point Likert-type coding.

Results: Both scoring methods were reliable (alpha > 0.88). However, deltaG was substantially lower (0.73) for the dichotomous coding of the GHQ-12 than for the Likert-type method (deltaG = 0.96), indicating that the dichotomous coding, although reliable, failed to discriminate between individuals.

Conclusion: Coefficient deltaG was shown to have decisive utility in distinguishing between the cross-sectional discrimination of two equally reliable scoring methods. Ferguson's delta has been neglected in discussions of questionnaire design and performance, perhaps because it has not been implemented in software and was restricted to questionnaires with dichotomous items, which are rare in health care research. It is suggested that the more general formula introduced here is reported as deltaG, to avoid the implication that items are dichotomously coded.

Show MeSH
Related in: MedlinePlus