Limits...
Thoracic surgeons' perception of frail behavior in videos of standardized patients.

Ferguson MK, Thompson K, Huisingh-Scheetz M, Farnan J, Hemmerich JA, Slawinski K, Acevedo J, Lee SM, Rojnica M, Small S - PLoS ONE (2014)

Bottom Line: We compared their performance to that of geriatrics specialists.More clinical years of experience and self-reported expertise were not associated with better inter-rater agreement.Ratings were less consistent for moderate degrees of frailty, suggesting that physicians require training to recognize early frailty.

View Article: PubMed Central - PubMed

Affiliation: Department of Surgery, The University of Chicago, Chicago, Illinois, United States of America.

ABSTRACT

Background: Frailty is a predictor of poor outcomes following many types of operations. We measured thoracic surgeons' accuracy in assessing patient frailty using videos of standardized patients demonstrating signs of physical frailty. We compared their performance to that of geriatrics specialists.

Methods: We developed an anchored scale for rating degree of frailty. Reference categories were assigned to 31 videos of standardized patients trained to exhibit five levels of activity ranging from "vigorous" to "frail." Following an explanation of frailty, thoracic surgeons and geriatrics specialists rated the videos. We evaluated inter-rater agreement and tested differences between ratings and reference categories. The influences of clinical specialty, clinical experience, and self-rated expertise were examined.

Results: Inter-rater rank correlation among all participants was high (Kendall's W 0.85) whereas exact agreement (Fleiss' kappa) was only moderate (0.47). Better inter-rater agreement was demonstrated for videos exhibiting extremes of behavior. Exact agreement was better for thoracic surgeons (n = 32) than geriatrics specialists (n = 9; p = 0.045), whereas rank correlation was similar for both groups. More clinical years of experience and self-reported expertise were not associated with better inter-rater agreement.

Conclusions: Videos of standardized patients exhibiting varying degrees of frailty are rated with internal consistency by thoracic surgeons as accurately as geriatrics specialists when referenced to an anchored scale. Ratings were less consistent for moderate degrees of frailty, suggesting that physicians require training to recognize early frailty. Such videos may be useful in assessing and teaching frailty recognition.

Show MeSH

Related in: MedlinePlus

Exact agreement with references scores by video category according to years of clinical experience.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4043843&req=5

pone-0098654-g003: Exact agreement with references scores by video category according to years of clinical experience.

Mentions: Overall agreement (exact matches to reference values) was somewhat higher for surgeons than for geriatricians (63.6% vs 58.1%; p = 0.090). Video categories importantly influenced inter-rater agreement. Overall agreement was substantially higher for vigorous and frail videos than for videos with intermediate levels of frailty or vigor (Figure 1; p<0.001). Agreement with reference values was higher for surgeons than for geriatricians for vigorous videos (p = 0.042). There were no differences between the specialties in their agreement for other video categories. Similar disparities in agreement rates for videos at the extremes compared to videos portraying intermediate activity levels were evident comparing experts to non-experts and experienced to less experienced raters (Figures 2 and 3).


Thoracic surgeons' perception of frail behavior in videos of standardized patients.

Ferguson MK, Thompson K, Huisingh-Scheetz M, Farnan J, Hemmerich JA, Slawinski K, Acevedo J, Lee SM, Rojnica M, Small S - PLoS ONE (2014)

Exact agreement with references scores by video category according to years of clinical experience.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4043843&req=5

pone-0098654-g003: Exact agreement with references scores by video category according to years of clinical experience.
Mentions: Overall agreement (exact matches to reference values) was somewhat higher for surgeons than for geriatricians (63.6% vs 58.1%; p = 0.090). Video categories importantly influenced inter-rater agreement. Overall agreement was substantially higher for vigorous and frail videos than for videos with intermediate levels of frailty or vigor (Figure 1; p<0.001). Agreement with reference values was higher for surgeons than for geriatricians for vigorous videos (p = 0.042). There were no differences between the specialties in their agreement for other video categories. Similar disparities in agreement rates for videos at the extremes compared to videos portraying intermediate activity levels were evident comparing experts to non-experts and experienced to less experienced raters (Figures 2 and 3).

Bottom Line: We compared their performance to that of geriatrics specialists.More clinical years of experience and self-reported expertise were not associated with better inter-rater agreement.Ratings were less consistent for moderate degrees of frailty, suggesting that physicians require training to recognize early frailty.

View Article: PubMed Central - PubMed

Affiliation: Department of Surgery, The University of Chicago, Chicago, Illinois, United States of America.

ABSTRACT

Background: Frailty is a predictor of poor outcomes following many types of operations. We measured thoracic surgeons' accuracy in assessing patient frailty using videos of standardized patients demonstrating signs of physical frailty. We compared their performance to that of geriatrics specialists.

Methods: We developed an anchored scale for rating degree of frailty. Reference categories were assigned to 31 videos of standardized patients trained to exhibit five levels of activity ranging from "vigorous" to "frail." Following an explanation of frailty, thoracic surgeons and geriatrics specialists rated the videos. We evaluated inter-rater agreement and tested differences between ratings and reference categories. The influences of clinical specialty, clinical experience, and self-rated expertise were examined.

Results: Inter-rater rank correlation among all participants was high (Kendall's W 0.85) whereas exact agreement (Fleiss' kappa) was only moderate (0.47). Better inter-rater agreement was demonstrated for videos exhibiting extremes of behavior. Exact agreement was better for thoracic surgeons (n = 32) than geriatrics specialists (n = 9; p = 0.045), whereas rank correlation was similar for both groups. More clinical years of experience and self-reported expertise were not associated with better inter-rater agreement.

Conclusions: Videos of standardized patients exhibiting varying degrees of frailty are rated with internal consistency by thoracic surgeons as accurately as geriatrics specialists when referenced to an anchored scale. Ratings were less consistent for moderate degrees of frailty, suggesting that physicians require training to recognize early frailty. Such videos may be useful in assessing and teaching frailty recognition.

Show MeSH
Related in: MedlinePlus