Limits...
Definitions and standardization of a new grading scheme for eyelid contour abnormalities after trichiasis surgery.

Gower EW, West SK, Cassard SD, Munoz BE, Harding JC, Merbs SL - PLoS Negl Trop Dis (2012)

Bottom Line: All six graders independently graded a Standardization Set of 75 eyelids, which included a roughly equal distribution across the severity scale, and weighted kappa scores were calculated.The weighted kappa for agreement between the two senior graders was 0.80 (95% CI: 0.71-0.89).Among the Standardization Set, agreement between the senior graders and the 4 new graders showed weighted kappa scores ranging from 0.60-0.80.

View Article: PubMed Central - PubMed

Affiliation: Department of Epidemiology and Prevention, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States of America. egower@wakehealth.edu

ABSTRACT

Background: Clear definitions of outcomes following trichiasis surgery are critical for planning program evaluations and for identifying ways to improve trichiasis surgery. Eyelid contour abnormality is an important adverse outcome of surgery; however, no standard method has been described to categorize eyelid contour abnormalities.

Methodology/principal findings: A classification system for eyelid contour abnormalities following surgery for trachomatous trichiasis was developed. To determine whether the grading was reproducible using the classification system, six-week postoperative photographs were reviewed by two senior graders to characterize severity of contour abnormalities. Sample photographs defining each contour abnormality category were compiled and used to train four new graders. All six graders independently graded a Standardization Set of 75 eyelids, which included a roughly equal distribution across the severity scale, and weighted kappa scores were calculated. Two hundred forty six-week postoperative photographs from an ongoing clinical trial were randomly selected for evaluating agreement across graders. Two months after initial grading, one grader regraded a subset of the 240 photographs to measure longer-term intra-observer agreement. The weighted kappa for agreement between the two senior graders was 0.80 (95% CI: 0.71-0.89). Among the Standardization Set, agreement between the senior graders and the 4 new graders showed weighted kappa scores ranging from 0.60-0.80. Among 240 eyes comprising the clinical trial dataset, agreement ranged from weighted kappa 0.70-0.71. Longer-term intra-observer agreement was weighted kappa 0.86 (95% CI: 0.80-0.92).

Conclusions/significance: The standard eyelid contour grading system we developed reproducibly delineates differing levels of contour abnormality. This grading system could be useful both for helping to evaluate trichiasis surgery outcomes in clinical trials and for evaluating trichiasis surgery programs.

Show MeSH

Related in: MedlinePlus

Standard photographs showing examples for each grading category.A and B) Mild eyelid contour abnormality C) Moderate abnormality based on height of deviation D) Moderate abnormality based on length of deviation. E) Severe abnormality based on height of deviation F) Severe abnormality based on length of deviation.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3383763&req=5

pntd-0001713-g001: Standard photographs showing examples for each grading category.A and B) Mild eyelid contour abnormality C) Moderate abnormality based on height of deviation D) Moderate abnormality based on length of deviation. E) Severe abnormality based on height of deviation F) Severe abnormality based on length of deviation.

Mentions: We compiled a set of standard photographs to illustrate abnormalities for each category. Photographs representing each category are provided in Figure 1. In addition, we created a Standardization Set of 75 eyelid photographs that were randomly selected with a roughly equal distribution across severity categories based on the clinical trial field grade. Agreement between graders 1 and 2 was measured for this 75-photograph standardization set. Graders 1 and 2 then met to adjudicate differences and to create gold standard grades for the Standardization Set. Several weeks later, grader 1 re-graded the Standardization Set.


Definitions and standardization of a new grading scheme for eyelid contour abnormalities after trichiasis surgery.

Gower EW, West SK, Cassard SD, Munoz BE, Harding JC, Merbs SL - PLoS Negl Trop Dis (2012)

Standard photographs showing examples for each grading category.A and B) Mild eyelid contour abnormality C) Moderate abnormality based on height of deviation D) Moderate abnormality based on length of deviation. E) Severe abnormality based on height of deviation F) Severe abnormality based on length of deviation.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3383763&req=5

pntd-0001713-g001: Standard photographs showing examples for each grading category.A and B) Mild eyelid contour abnormality C) Moderate abnormality based on height of deviation D) Moderate abnormality based on length of deviation. E) Severe abnormality based on height of deviation F) Severe abnormality based on length of deviation.
Mentions: We compiled a set of standard photographs to illustrate abnormalities for each category. Photographs representing each category are provided in Figure 1. In addition, we created a Standardization Set of 75 eyelid photographs that were randomly selected with a roughly equal distribution across severity categories based on the clinical trial field grade. Agreement between graders 1 and 2 was measured for this 75-photograph standardization set. Graders 1 and 2 then met to adjudicate differences and to create gold standard grades for the Standardization Set. Several weeks later, grader 1 re-graded the Standardization Set.

Bottom Line: All six graders independently graded a Standardization Set of 75 eyelids, which included a roughly equal distribution across the severity scale, and weighted kappa scores were calculated.The weighted kappa for agreement between the two senior graders was 0.80 (95% CI: 0.71-0.89).Among the Standardization Set, agreement between the senior graders and the 4 new graders showed weighted kappa scores ranging from 0.60-0.80.

View Article: PubMed Central - PubMed

Affiliation: Department of Epidemiology and Prevention, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States of America. egower@wakehealth.edu

ABSTRACT

Background: Clear definitions of outcomes following trichiasis surgery are critical for planning program evaluations and for identifying ways to improve trichiasis surgery. Eyelid contour abnormality is an important adverse outcome of surgery; however, no standard method has been described to categorize eyelid contour abnormalities.

Methodology/principal findings: A classification system for eyelid contour abnormalities following surgery for trachomatous trichiasis was developed. To determine whether the grading was reproducible using the classification system, six-week postoperative photographs were reviewed by two senior graders to characterize severity of contour abnormalities. Sample photographs defining each contour abnormality category were compiled and used to train four new graders. All six graders independently graded a Standardization Set of 75 eyelids, which included a roughly equal distribution across the severity scale, and weighted kappa scores were calculated. Two hundred forty six-week postoperative photographs from an ongoing clinical trial were randomly selected for evaluating agreement across graders. Two months after initial grading, one grader regraded a subset of the 240 photographs to measure longer-term intra-observer agreement. The weighted kappa for agreement between the two senior graders was 0.80 (95% CI: 0.71-0.89). Among the Standardization Set, agreement between the senior graders and the 4 new graders showed weighted kappa scores ranging from 0.60-0.80. Among 240 eyes comprising the clinical trial dataset, agreement ranged from weighted kappa 0.70-0.71. Longer-term intra-observer agreement was weighted kappa 0.86 (95% CI: 0.80-0.92).

Conclusions/significance: The standard eyelid contour grading system we developed reproducibly delineates differing levels of contour abnormality. This grading system could be useful both for helping to evaluate trichiasis surgery outcomes in clinical trials and for evaluating trichiasis surgery programs.

Show MeSH
Related in: MedlinePlus