Limits...
Insightful Practice: a robust measure of medical students' professional response to feedback on their performance.

Murphy D, Aitchison P, Hernandez Santiago V, Davey P, Mires G, Nathwani D - BMC Med Educ (2015)

Bottom Line: Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training.Secondary outcome measures were the reaction of participants and their self-reported behavioural change.The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'.

View Article: PubMed Central - PubMed

Affiliation: Quality, Safety and Informatics Research Group, Population Health Sciences, University of Dundee, Mackenzie Building, Kirsty Semple Way, Dundee, DD2 4BF, UK. d.y.murphy@dundee.ac.uk.

ABSTRACT

Background: Healthcare professionals need to show accountability, responsibility and appropriate response to audit feedback. Assessment of Insightful Practice (engagement, insight and appropriate action for improvement) has been shown to offer a robust system, in general practice, to identify concerns in doctors' response to independent feedback. This study researched the system's utility in medical undergraduates.

Setting and participants: 28 fourth year medical students reflected on their performance feedback. Reflection was supported by a staff coach. Students' portfolios were divided into two groups (n = 14). Group 1 students were assessed by three staff assessors (calibrated using group training) and Group 2 students' portfolios were assessed by three staff assessors (un-calibrated by one-to-one training). Assessments were by blinded web-based exercise and assessors were senior Medical School staff.

Design: Case series with mixed qualitative and quantitative methods. A feedback dataset was specified as (1) student-specific End-of-Block Clinical Feedback, (2) other available Medical School assessment data and, (3) an assessment of students' identification of prescribing errors. Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training. One-to-one interviews explored participants' experiences.

Main outcome measures: The primary outcome measure was inter-rater reliability of assessment of students' Insightful Practice. Secondary outcome measures were the reaction of participants and their self-reported behavioural change.

Results: The method offered a feasible and highly reliable global assessment for calibrated assessors, G (inter-rater reliability) > 0.8 (two assessors), but not un-calibrated assessors G < 0.31. Calibrated assessment proved an acceptable basis to enhance feedback and identify concern in professionalism. Students reported increased awareness in teamwork and in the importance of heeding advice. Coaches reported improvement in their feedback skills and commitment to improving the quality of student feedback.

Conclusions: Insightful practice offers a reliable and feasible method to evaluate medical undergraduates' professional response to their training feedback. The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'.

No MeSH data available.


Cycle of insightful practice
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
getmorefigures.php?uid=PMC4522119&req=5

Fig1: Cycle of insightful practice

Mentions: The proposed role of insightful practice to act as the hub within a continuous cycle to generate, monitor and maintain objective evidence of personal responsibility and accountability for quality improvement has already been demonstrated (Fig 1).Fig. 1


Insightful Practice: a robust measure of medical students' professional response to feedback on their performance.

Murphy D, Aitchison P, Hernandez Santiago V, Davey P, Mires G, Nathwani D - BMC Med Educ (2015)

Cycle of insightful practice
© Copyright Policy - open-access
Related In: Results  -  Collection

License 1 - License 2
Show All Figures
getmorefigures.php?uid=PMC4522119&req=5

Fig1: Cycle of insightful practice
Mentions: The proposed role of insightful practice to act as the hub within a continuous cycle to generate, monitor and maintain objective evidence of personal responsibility and accountability for quality improvement has already been demonstrated (Fig 1).Fig. 1

Bottom Line: Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training.Secondary outcome measures were the reaction of participants and their self-reported behavioural change.The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'.

View Article: PubMed Central - PubMed

Affiliation: Quality, Safety and Informatics Research Group, Population Health Sciences, University of Dundee, Mackenzie Building, Kirsty Semple Way, Dundee, DD2 4BF, UK. d.y.murphy@dundee.ac.uk.

ABSTRACT

Background: Healthcare professionals need to show accountability, responsibility and appropriate response to audit feedback. Assessment of Insightful Practice (engagement, insight and appropriate action for improvement) has been shown to offer a robust system, in general practice, to identify concerns in doctors' response to independent feedback. This study researched the system's utility in medical undergraduates.

Setting and participants: 28 fourth year medical students reflected on their performance feedback. Reflection was supported by a staff coach. Students' portfolios were divided into two groups (n = 14). Group 1 students were assessed by three staff assessors (calibrated using group training) and Group 2 students' portfolios were assessed by three staff assessors (un-calibrated by one-to-one training). Assessments were by blinded web-based exercise and assessors were senior Medical School staff.

Design: Case series with mixed qualitative and quantitative methods. A feedback dataset was specified as (1) student-specific End-of-Block Clinical Feedback, (2) other available Medical School assessment data and, (3) an assessment of students' identification of prescribing errors. Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training. One-to-one interviews explored participants' experiences.

Main outcome measures: The primary outcome measure was inter-rater reliability of assessment of students' Insightful Practice. Secondary outcome measures were the reaction of participants and their self-reported behavioural change.

Results: The method offered a feasible and highly reliable global assessment for calibrated assessors, G (inter-rater reliability) > 0.8 (two assessors), but not un-calibrated assessors G < 0.31. Calibrated assessment proved an acceptable basis to enhance feedback and identify concern in professionalism. Students reported increased awareness in teamwork and in the importance of heeding advice. Coaches reported improvement in their feedback skills and commitment to improving the quality of student feedback.

Conclusions: Insightful practice offers a reliable and feasible method to evaluate medical undergraduates' professional response to their training feedback. The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'.

No MeSH data available.