Limits...
Beyond readability: investigating coherence of clinical text for consumers.

Smith CA, Hetzel S, Dalrymple P, Keselman A - J. Med. Internet Res. (2011)

Bottom Line: The need for efficient ways to assess consumer-oriented health texts and the availability of computationally supported tools led us to explore the effect of various text characteristics on readers' understanding of health texts, as well as to develop novel approaches to assessing these characteristics.However, no difference was seen between (Original+Dictionary) and Vocabulary (P=.36) nor Coherent and Vocabulary (P=.62).Since comprehensibility of clinical text appears difficult to automate, informatics support tools may most productively support the health care professionals tasked with making clinical information understandable to patients.

View Article: PubMed Central - HTML - PubMed

Affiliation: School of Library and Information Studies, University of Wisconsin-Madison, Madison, WI 53706, United States. casmith24@wisc.edu

ABSTRACT

Background: A basic tenet of consumer health informatics is that understandable health resources empower the public. Text comprehension holds great promise for helping to characterize consumer problems in understanding health texts. The need for efficient ways to assess consumer-oriented health texts and the availability of computationally supported tools led us to explore the effect of various text characteristics on readers' understanding of health texts, as well as to develop novel approaches to assessing these characteristics.

Objective: The goal of this study was to compare the impact of two different approaches to enhancing readability, and three interventions, on individuals' comprehension of short, complex passages of health text.

Methods: Participants were 80 university staff, faculty, or students. Each participant was asked to "retell" the content of two health texts: one a clinical trial in the domain of diabetes mellitus, and the other typical Visit Notes. These texts were transformed for the intervention arms of the study. Two interventions provided terminology support via (1) standard dictionary or (2) contextualized vocabulary definitions. The third intervention provided coherence improvement. We assessed participants' comprehension of the clinical texts through propositional analysis, an open-ended questionnaire, and analysis of the number of errors made.

Results: For the clinical trial text, the effect of text condition was not significant in any of the comparisons, suggesting no differences in recall, despite the varying levels of support (P=.84). For the Visit Note, however, the difference in the median total propositions recalled between the Coherent and the (Original+Dictionary) conditions was significant (P=.04). This suggests that participants in the Coherent condition recalled more of the original Visit Notes content than did participants in the Original and the Dictionary conditions combined. However, no difference was seen between (Original+Dictionary) and Vocabulary (P=.36) nor Coherent and Vocabulary (P=.62). No statistically significant effect of any document transformation was found either in the open-ended questionnaire (clinical trial: P=.86, Visit Note: P=.20) or in the error rate (clinical trial: P=.47, Visit Note: P=.25). However, post hoc power analysis suggested that increasing the sample size by approximately 6 participants per condition would result in a significant difference for the Visit Note, but not for the clinical trial text.

Conclusions: Statistically, the results of this study attest that improving coherence has a small effect on consumer comprehension of clinical text, but the task is extremely labor intensive and not scalable. Further research is needed using texts from more diverse clinical domains and more heterogeneous participants, including actual patients. Since comprehensibility of clinical text appears difficult to automate, informatics support tools may most productively support the health care professionals tasked with making clinical information understandable to patients.

Show MeSH

Related in: MedlinePlus

Clinical trial document with coherence enhancement.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC3278090&req=5

figure3: Clinical trial document with coherence enhancement.

Mentions: We first segmented this text into units of analysis, usually complete sentences. In some cases, complex sentences were divided into propositions, keeping intact phrases beginning with words such as “therefore” or “because.” Next, we identified coherence gaps, defined as places where an inference was needed to comprehend each sentence on the basis of preceding sentences. Information was then added to the text, either by supplementing existing sentences or by adding new sentences that contained contextualized explanations. Examples of such added information include a missing background concept—for example, an explanation of the dangers of hypoglycemia—or the rationale behind the assessment procedure—for example, explaining the need to have good methods for measuring liver glycogen metabolism. Additionally, to make the clinical trial’s research objectives more obvious, information about the purpose of the trial was rearranged from its original location so that it appeared in the opening sentence of the transformed document. Finally, to ensure local coherence, we checked the final text to ensure that the referents of pronouns were explicit. The coherence-transformed clinical trial text appears in Figure 3.


Beyond readability: investigating coherence of clinical text for consumers.

Smith CA, Hetzel S, Dalrymple P, Keselman A - J. Med. Internet Res. (2011)

Clinical trial document with coherence enhancement.
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC3278090&req=5

figure3: Clinical trial document with coherence enhancement.
Mentions: We first segmented this text into units of analysis, usually complete sentences. In some cases, complex sentences were divided into propositions, keeping intact phrases beginning with words such as “therefore” or “because.” Next, we identified coherence gaps, defined as places where an inference was needed to comprehend each sentence on the basis of preceding sentences. Information was then added to the text, either by supplementing existing sentences or by adding new sentences that contained contextualized explanations. Examples of such added information include a missing background concept—for example, an explanation of the dangers of hypoglycemia—or the rationale behind the assessment procedure—for example, explaining the need to have good methods for measuring liver glycogen metabolism. Additionally, to make the clinical trial’s research objectives more obvious, information about the purpose of the trial was rearranged from its original location so that it appeared in the opening sentence of the transformed document. Finally, to ensure local coherence, we checked the final text to ensure that the referents of pronouns were explicit. The coherence-transformed clinical trial text appears in Figure 3.

Bottom Line: The need for efficient ways to assess consumer-oriented health texts and the availability of computationally supported tools led us to explore the effect of various text characteristics on readers' understanding of health texts, as well as to develop novel approaches to assessing these characteristics.However, no difference was seen between (Original+Dictionary) and Vocabulary (P=.36) nor Coherent and Vocabulary (P=.62).Since comprehensibility of clinical text appears difficult to automate, informatics support tools may most productively support the health care professionals tasked with making clinical information understandable to patients.

View Article: PubMed Central - HTML - PubMed

Affiliation: School of Library and Information Studies, University of Wisconsin-Madison, Madison, WI 53706, United States. casmith24@wisc.edu

ABSTRACT

Background: A basic tenet of consumer health informatics is that understandable health resources empower the public. Text comprehension holds great promise for helping to characterize consumer problems in understanding health texts. The need for efficient ways to assess consumer-oriented health texts and the availability of computationally supported tools led us to explore the effect of various text characteristics on readers' understanding of health texts, as well as to develop novel approaches to assessing these characteristics.

Objective: The goal of this study was to compare the impact of two different approaches to enhancing readability, and three interventions, on individuals' comprehension of short, complex passages of health text.

Methods: Participants were 80 university staff, faculty, or students. Each participant was asked to "retell" the content of two health texts: one a clinical trial in the domain of diabetes mellitus, and the other typical Visit Notes. These texts were transformed for the intervention arms of the study. Two interventions provided terminology support via (1) standard dictionary or (2) contextualized vocabulary definitions. The third intervention provided coherence improvement. We assessed participants' comprehension of the clinical texts through propositional analysis, an open-ended questionnaire, and analysis of the number of errors made.

Results: For the clinical trial text, the effect of text condition was not significant in any of the comparisons, suggesting no differences in recall, despite the varying levels of support (P=.84). For the Visit Note, however, the difference in the median total propositions recalled between the Coherent and the (Original+Dictionary) conditions was significant (P=.04). This suggests that participants in the Coherent condition recalled more of the original Visit Notes content than did participants in the Original and the Dictionary conditions combined. However, no difference was seen between (Original+Dictionary) and Vocabulary (P=.36) nor Coherent and Vocabulary (P=.62). No statistically significant effect of any document transformation was found either in the open-ended questionnaire (clinical trial: P=.86, Visit Note: P=.20) or in the error rate (clinical trial: P=.47, Visit Note: P=.25). However, post hoc power analysis suggested that increasing the sample size by approximately 6 participants per condition would result in a significant difference for the Visit Note, but not for the clinical trial text.

Conclusions: Statistically, the results of this study attest that improving coherence has a small effect on consumer comprehension of clinical text, but the task is extremely labor intensive and not scalable. Further research is needed using texts from more diverse clinical domains and more heterogeneous participants, including actual patients. Since comprehensibility of clinical text appears difficult to automate, informatics support tools may most productively support the health care professionals tasked with making clinical information understandable to patients.

Show MeSH
Related in: MedlinePlus