Limits...
Designing questionnaires: healthcare survey to compare two different response scales.

Dell-Kuster S, Sanjuan E, Todorov A, Weber H, Heberer M, Rosenthal R - BMC Med Res Methodol (2014)

Bottom Line: The scales were compared concerning ceiling effect, internal consistency (Cronbach's alpha), individual item answers (Spearman's rank correlation), and concerning overall satisfaction by calculating an overall percentage score (sum of all answers related to the maximum possible sum).Floor and ceiling effect were high on both response scales, but more pronounced on the LS than on the NS.Since the longer scale did not substantially reduce the ceiling effect, the type of questions rather than the type of answering scale could be addressed with a focus on specific questions about concrete situations instead of general questions.

View Article: PubMed Central - HTML - PubMed

Affiliation: Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Hebelstrasse 10, 4031 Basel, Switzerland. Salome.Dell-Kuster@usb.ch.

ABSTRACT

Background: A widely discussed design issue in patient satisfaction questionnaires is the optimal length and labelling of the answering scale. The aim of the present study was to compare intra-individually the answers on two response scales to five general questions evaluating patients' perception of hospital care.

Methods: Between November 2011 and January 2012, all in-hospital patients at a Swiss University Hospital received a patient satisfaction questionnaire on an adjectival scale with three to four labelled categories (LS) and five redundant questions displayed on an 11-point end-anchored numeric scale (NS). The scales were compared concerning ceiling effect, internal consistency (Cronbach's alpha), individual item answers (Spearman's rank correlation), and concerning overall satisfaction by calculating an overall percentage score (sum of all answers related to the maximum possible sum).

Results: The response rate was 41% (2957/7158), of which 2400 (81%) completely filled out all questions. Baseline characteristics of the responders and non-responders were similar. Floor and ceiling effect were high on both response scales, but more pronounced on the LS than on the NS. Cronbach's alpha was higher on the NS than on the LS. There was a strong individual item correlation between both answering scales in questions regarding the intent to return, quality of treatment and the judgement whether the patient was treated with respect and dignity, but a lower correlation concerning satisfactory information transfer by physicians or nurses, where only three categories were available in the LS. The overall percentage score showed a comparable distribution, but with a wider spread of lower satisfaction in the NS.

Conclusions: Since the longer scale did not substantially reduce the ceiling effect, the type of questions rather than the type of answering scale could be addressed with a focus on specific questions about concrete situations instead of general questions. Moreover, the low correlation in questions about information provision suggests that only three possible response choices are insufficient. Further investigations are needed to find a more sensitive scale discriminating high-end ratings. Otherwise, a longitudinal within-hospital or a cross-sectional between-hospital comparison of patient care is questionable.

Show MeSH
Correlation of the answers of patients on both scales - Question quality of treatment. Scatterplot for comparison of the answers of each patient on two different answering scales with an overlaid smoother for each item separately. The dots are proportional to the frequency of the corresponding combination.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4126910&req=5

Figure 6: Correlation of the answers of patients on both scales - Question quality of treatment. Scatterplot for comparison of the answers of each patient on two different answering scales with an overlaid smoother for each item separately. The dots are proportional to the frequency of the corresponding combination.

Mentions: The results of the correlation of the individual item answers between both response scales are displayed in Table 3 (all questions) and Figures 6 and 7 for two representative questions (quality of treatment and quality of information provided by physicians; Webappendix: Additional files 4, 5 and 6 for the remaining three questions). There was a strong correlation between the response scales regarding questions about intent to return, quality of treatment and judgement whether the patient was treated with respect and dignity, but a lower correlation concerning satisfactory information transfer by physicians or nurses.


Designing questionnaires: healthcare survey to compare two different response scales.

Dell-Kuster S, Sanjuan E, Todorov A, Weber H, Heberer M, Rosenthal R - BMC Med Res Methodol (2014)

Correlation of the answers of patients on both scales - Question quality of treatment. Scatterplot for comparison of the answers of each patient on two different answering scales with an overlaid smoother for each item separately. The dots are proportional to the frequency of the corresponding combination.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4126910&req=5

Figure 6: Correlation of the answers of patients on both scales - Question quality of treatment. Scatterplot for comparison of the answers of each patient on two different answering scales with an overlaid smoother for each item separately. The dots are proportional to the frequency of the corresponding combination.
Mentions: The results of the correlation of the individual item answers between both response scales are displayed in Table 3 (all questions) and Figures 6 and 7 for two representative questions (quality of treatment and quality of information provided by physicians; Webappendix: Additional files 4, 5 and 6 for the remaining three questions). There was a strong correlation between the response scales regarding questions about intent to return, quality of treatment and judgement whether the patient was treated with respect and dignity, but a lower correlation concerning satisfactory information transfer by physicians or nurses.

Bottom Line: The scales were compared concerning ceiling effect, internal consistency (Cronbach's alpha), individual item answers (Spearman's rank correlation), and concerning overall satisfaction by calculating an overall percentage score (sum of all answers related to the maximum possible sum).Floor and ceiling effect were high on both response scales, but more pronounced on the LS than on the NS.Since the longer scale did not substantially reduce the ceiling effect, the type of questions rather than the type of answering scale could be addressed with a focus on specific questions about concrete situations instead of general questions.

View Article: PubMed Central - HTML - PubMed

Affiliation: Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Hebelstrasse 10, 4031 Basel, Switzerland. Salome.Dell-Kuster@usb.ch.

ABSTRACT

Background: A widely discussed design issue in patient satisfaction questionnaires is the optimal length and labelling of the answering scale. The aim of the present study was to compare intra-individually the answers on two response scales to five general questions evaluating patients' perception of hospital care.

Methods: Between November 2011 and January 2012, all in-hospital patients at a Swiss University Hospital received a patient satisfaction questionnaire on an adjectival scale with three to four labelled categories (LS) and five redundant questions displayed on an 11-point end-anchored numeric scale (NS). The scales were compared concerning ceiling effect, internal consistency (Cronbach's alpha), individual item answers (Spearman's rank correlation), and concerning overall satisfaction by calculating an overall percentage score (sum of all answers related to the maximum possible sum).

Results: The response rate was 41% (2957/7158), of which 2400 (81%) completely filled out all questions. Baseline characteristics of the responders and non-responders were similar. Floor and ceiling effect were high on both response scales, but more pronounced on the LS than on the NS. Cronbach's alpha was higher on the NS than on the LS. There was a strong individual item correlation between both answering scales in questions regarding the intent to return, quality of treatment and the judgement whether the patient was treated with respect and dignity, but a lower correlation concerning satisfactory information transfer by physicians or nurses, where only three categories were available in the LS. The overall percentage score showed a comparable distribution, but with a wider spread of lower satisfaction in the NS.

Conclusions: Since the longer scale did not substantially reduce the ceiling effect, the type of questions rather than the type of answering scale could be addressed with a focus on specific questions about concrete situations instead of general questions. Moreover, the low correlation in questions about information provision suggests that only three possible response choices are insufficient. Further investigations are needed to find a more sensitive scale discriminating high-end ratings. Otherwise, a longitudinal within-hospital or a cross-sectional between-hospital comparison of patient care is questionable.

Show MeSH