Limits...
A nomogram to predict the probability of passing the American Board of Internal Medicine examination.

Brateanu A, Yu C, Kattan MW, Olender J, Nielsen C - Med Educ Online (2012)

Bottom Line: Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination.A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents.This may guide program directors' decision-making related to program curriculum and advice given to individual residents regarding board preparation.

View Article: PubMed Central - PubMed

Affiliation: Department of Medicine, Cleveland Clinic, Cleveland, Ohio 44195, USA. abratean@ccf.org

ABSTRACT

Background: Although the American Board of Internal Medicine (ABIM) certification is valued as a reflection of physicians' experience, education, and expertise, limited methods exist to predict performance in the examination.

Purpose: The objective of this study was to develop and validate a predictive tool based on variables common to all residency programs, regarding the probability of an internal medicine graduate passing the ABIM certification examination.

Methods: The development cohort was obtained from the files of the Cleveland Clinic internal medicine residents who began training between 2004 and 2008. A multivariable logistic regression model was built to predict the ABIM passing rate. The model was represented as a nomogram, which was internally validated with bootstrap resamples. The external validation was done retrospectively on a cohort of residents who graduated from two other independent internal medicine residency programs between 2007 and 2011.

Results: Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination. The final nomogram included four predictors: In-Training Examination (ITE) scores in postgraduate year (PGY) 1, 2, and 3, and the number of months of overnight calls in the last 6 months of residency. The nomogram achieved a concordance index (CI) of 0.98 after correcting for over-fitting bias and allowed for the determination of an estimated probability of passing the ABIM exam. Of the 126 graduates from two other residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The nomogram CI in the external validation cohort was 0.94, suggesting outstanding discrimination.

Conclusions: A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents. This may guide program directors' decision-making related to program curriculum and advice given to individual residents regarding board preparation.

Show MeSH

Related in: MedlinePlus

Calibration plot for predicted and observed passing ABIM examination. Residents in the validation cohort were divided in five quintiles based on the predicted probability. The 45-degree straight line represents ideal agreement between actual and predicted probability. The vertical bars represent the 95% CI of the actual probability. Note that the points for the three highest quintiles were overlapping together in the plot because of ignorable differences.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3475012&req=5

Figure 0003: Calibration plot for predicted and observed passing ABIM examination. Residents in the validation cohort were divided in five quintiles based on the predicted probability. The 45-degree straight line represents ideal agreement between actual and predicted probability. The vertical bars represent the 95% CI of the actual probability. Note that the points for the three highest quintiles were overlapping together in the plot because of ignorable differences.

Mentions: Of the 126 graduates from the other two residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The calibration plot was generated by grouping the validation cohort into five subgroups, sorted by the predicting passing probabilities and plotting the mean predicted probabilities against the actual fraction of residents within each quintile (Fig. 3). The nomogram CI in the validation cohort was 0.94 (95% CI, 0.90–0.99). When compared with each of the four individual predictors included in the validation set, the nomogram performed better than the ITE 1, 2, 3 scores and the number of call months in the last 6 months of the residency. The ITE 3 score was the most important predictor, which was reflected in the nomogram as well (Table 3). The correspondence between the actual passing rate and the nomogram predictions suggests a good calibration of the nomogram in the validation cohort. The model has a very good prediction for graduates with high likelihood to pass the exam and slightly overestimates the passing rate for those with low likelihood.


A nomogram to predict the probability of passing the American Board of Internal Medicine examination.

Brateanu A, Yu C, Kattan MW, Olender J, Nielsen C - Med Educ Online (2012)

Calibration plot for predicted and observed passing ABIM examination. Residents in the validation cohort were divided in five quintiles based on the predicted probability. The 45-degree straight line represents ideal agreement between actual and predicted probability. The vertical bars represent the 95% CI of the actual probability. Note that the points for the three highest quintiles were overlapping together in the plot because of ignorable differences.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3475012&req=5

Figure 0003: Calibration plot for predicted and observed passing ABIM examination. Residents in the validation cohort were divided in five quintiles based on the predicted probability. The 45-degree straight line represents ideal agreement between actual and predicted probability. The vertical bars represent the 95% CI of the actual probability. Note that the points for the three highest quintiles were overlapping together in the plot because of ignorable differences.
Mentions: Of the 126 graduates from the other two residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The calibration plot was generated by grouping the validation cohort into five subgroups, sorted by the predicting passing probabilities and plotting the mean predicted probabilities against the actual fraction of residents within each quintile (Fig. 3). The nomogram CI in the validation cohort was 0.94 (95% CI, 0.90–0.99). When compared with each of the four individual predictors included in the validation set, the nomogram performed better than the ITE 1, 2, 3 scores and the number of call months in the last 6 months of the residency. The ITE 3 score was the most important predictor, which was reflected in the nomogram as well (Table 3). The correspondence between the actual passing rate and the nomogram predictions suggests a good calibration of the nomogram in the validation cohort. The model has a very good prediction for graduates with high likelihood to pass the exam and slightly overestimates the passing rate for those with low likelihood.

Bottom Line: Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination.A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents.This may guide program directors' decision-making related to program curriculum and advice given to individual residents regarding board preparation.

View Article: PubMed Central - PubMed

Affiliation: Department of Medicine, Cleveland Clinic, Cleveland, Ohio 44195, USA. abratean@ccf.org

ABSTRACT

Background: Although the American Board of Internal Medicine (ABIM) certification is valued as a reflection of physicians' experience, education, and expertise, limited methods exist to predict performance in the examination.

Purpose: The objective of this study was to develop and validate a predictive tool based on variables common to all residency programs, regarding the probability of an internal medicine graduate passing the ABIM certification examination.

Methods: The development cohort was obtained from the files of the Cleveland Clinic internal medicine residents who began training between 2004 and 2008. A multivariable logistic regression model was built to predict the ABIM passing rate. The model was represented as a nomogram, which was internally validated with bootstrap resamples. The external validation was done retrospectively on a cohort of residents who graduated from two other independent internal medicine residency programs between 2007 and 2011.

Results: Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination. The final nomogram included four predictors: In-Training Examination (ITE) scores in postgraduate year (PGY) 1, 2, and 3, and the number of months of overnight calls in the last 6 months of residency. The nomogram achieved a concordance index (CI) of 0.98 after correcting for over-fitting bias and allowed for the determination of an estimated probability of passing the ABIM exam. Of the 126 graduates from two other residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The nomogram CI in the external validation cohort was 0.94, suggesting outstanding discrimination.

Conclusions: A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents. This may guide program directors' decision-making related to program curriculum and advice given to individual residents regarding board preparation.

Show MeSH
Related in: MedlinePlus