Limits...
Interrater reliability of the mind map assessment rubric in a cohort of medical students.

D'Antoni AV, Zipp GP, Olson VG - BMC Med Educ (2009)

Bottom Line: The high ICC value for total mind map score indicates strong MMAR interrater reliability.Pictures and colors demonstrated moderate to strong interrater reliability.However, further research on the validity and reliability of the MMAR is necessary.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Graduate Programs in Health Sciences, School of Health and Medical Sciences, Seton Hall University, South Orange, NJ 07079, USA. avdantoni@si.rr.com

ABSTRACT

Background: Learning strategies are thinking tools that students can use to actively acquire information. Examples of learning strategies include mnemonics, charts, and maps. One strategy that may help students master the tsunami of information presented in medical school is the mind map learning strategy. Currently, there is no valid and reliable rubric to grade mind maps and this may contribute to their underutilization in medicine. Because concept maps and mind maps engage learners similarly at a metacognitive level, a valid and reliable concept map assessment scoring system was adapted to form the mind map assessment rubric (MMAR). The MMAR can assess mind map depth based upon concept-links, cross-links, hierarchies, examples, pictures, and colors. The purpose of this study was to examine interrater reliability of the MMAR.

Methods: This exploratory study was conducted at a US medical school as part of a larger investigation on learning strategies. Sixty-six (N = 66) first-year medical students were given a 394-word text passage followed by a 30-minute presentation on mind mapping. After the presentation, subjects were again given the text passage and instructed to create mind maps based upon the passage. The mind maps were collected and independently scored using the MMAR by 3 examiners. Interrater reliability was measured using the intraclass correlation coefficient (ICC) statistic. Statistics were calculated using SPSS version 12.0 (Chicago, IL).

Results: Analysis of the mind maps revealed the following: concept-links ICC = .05 (95% CI, -.42 to .38), cross-links ICC = .58 (95% CI, .37 to .73), hierarchies ICC = .23 (95% CI, -.15 to .50), examples ICC = .53 (95% CI, .29 to .69), pictures ICC = .86 (95% CI, .79 to .91), colors ICC = .73 (95% CI, .59 to .82), and total score ICC = .86 (95% CI, .79 to .91).

Conclusion: The high ICC value for total mind map score indicates strong MMAR interrater reliability. Pictures and colors demonstrated moderate to strong interrater reliability. We conclude that the MMAR may be a valid and reliable tool to assess mind maps in medicine. However, further research on the validity and reliability of the MMAR is necessary.

Show MeSH

Related in: MedlinePlus

Student mind map. An example of a high-scoring mind map from one of the medical students in this study. AVD assigned this mind map a total score of 400, GPZ assigned it a total score of 337, and VGO assigned it a total score of 377. The average total score of this mind map, based on all 3 examiners, was 371.33. Note the hierarchical organization of the mind map and the effective use of pictures and colors. In addition, this map contains numerous cross-links, which resulted in higher scores.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2683832&req=5

Figure 1: Student mind map. An example of a high-scoring mind map from one of the medical students in this study. AVD assigned this mind map a total score of 400, GPZ assigned it a total score of 337, and VGO assigned it a total score of 377. The average total score of this mind map, based on all 3 examiners, was 371.33. Note the hierarchical organization of the mind map and the effective use of pictures and colors. In addition, this map contains numerous cross-links, which resulted in higher scores.

Mentions: Like concept maps, another mapping strategy that relies on student interpretation and understanding is the mind map strategy. This learning strategy has not been widely used in medical education despite recent research suggesting that mind mapping improves long-term memory in medical students [11]. Mind mapping was developed by Tony Buzan [12] and inspired by the notebooks of Leonardo da Vinci [13]. Unlike most learners' notes, da Vinci's notes were not linear but elliptical–he used pictures and text to illustrate ideas and often connected different concepts on the same page. Mind maps, like da Vinci's notes, are multi-sensory tools that use visuospatial orientation to integrate information, and consequently, help students organize and retain information [14]. An example of a mind map created by a medical student in this study is shown in Figure 1. Other examples of mind maps created by health professional students have been published elsewhere [15,16].


Interrater reliability of the mind map assessment rubric in a cohort of medical students.

D'Antoni AV, Zipp GP, Olson VG - BMC Med Educ (2009)

Student mind map. An example of a high-scoring mind map from one of the medical students in this study. AVD assigned this mind map a total score of 400, GPZ assigned it a total score of 337, and VGO assigned it a total score of 377. The average total score of this mind map, based on all 3 examiners, was 371.33. Note the hierarchical organization of the mind map and the effective use of pictures and colors. In addition, this map contains numerous cross-links, which resulted in higher scores.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2683832&req=5

Figure 1: Student mind map. An example of a high-scoring mind map from one of the medical students in this study. AVD assigned this mind map a total score of 400, GPZ assigned it a total score of 337, and VGO assigned it a total score of 377. The average total score of this mind map, based on all 3 examiners, was 371.33. Note the hierarchical organization of the mind map and the effective use of pictures and colors. In addition, this map contains numerous cross-links, which resulted in higher scores.
Mentions: Like concept maps, another mapping strategy that relies on student interpretation and understanding is the mind map strategy. This learning strategy has not been widely used in medical education despite recent research suggesting that mind mapping improves long-term memory in medical students [11]. Mind mapping was developed by Tony Buzan [12] and inspired by the notebooks of Leonardo da Vinci [13]. Unlike most learners' notes, da Vinci's notes were not linear but elliptical–he used pictures and text to illustrate ideas and often connected different concepts on the same page. Mind maps, like da Vinci's notes, are multi-sensory tools that use visuospatial orientation to integrate information, and consequently, help students organize and retain information [14]. An example of a mind map created by a medical student in this study is shown in Figure 1. Other examples of mind maps created by health professional students have been published elsewhere [15,16].

Bottom Line: The high ICC value for total mind map score indicates strong MMAR interrater reliability.Pictures and colors demonstrated moderate to strong interrater reliability.However, further research on the validity and reliability of the MMAR is necessary.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Graduate Programs in Health Sciences, School of Health and Medical Sciences, Seton Hall University, South Orange, NJ 07079, USA. avdantoni@si.rr.com

ABSTRACT

Background: Learning strategies are thinking tools that students can use to actively acquire information. Examples of learning strategies include mnemonics, charts, and maps. One strategy that may help students master the tsunami of information presented in medical school is the mind map learning strategy. Currently, there is no valid and reliable rubric to grade mind maps and this may contribute to their underutilization in medicine. Because concept maps and mind maps engage learners similarly at a metacognitive level, a valid and reliable concept map assessment scoring system was adapted to form the mind map assessment rubric (MMAR). The MMAR can assess mind map depth based upon concept-links, cross-links, hierarchies, examples, pictures, and colors. The purpose of this study was to examine interrater reliability of the MMAR.

Methods: This exploratory study was conducted at a US medical school as part of a larger investigation on learning strategies. Sixty-six (N = 66) first-year medical students were given a 394-word text passage followed by a 30-minute presentation on mind mapping. After the presentation, subjects were again given the text passage and instructed to create mind maps based upon the passage. The mind maps were collected and independently scored using the MMAR by 3 examiners. Interrater reliability was measured using the intraclass correlation coefficient (ICC) statistic. Statistics were calculated using SPSS version 12.0 (Chicago, IL).

Results: Analysis of the mind maps revealed the following: concept-links ICC = .05 (95% CI, -.42 to .38), cross-links ICC = .58 (95% CI, .37 to .73), hierarchies ICC = .23 (95% CI, -.15 to .50), examples ICC = .53 (95% CI, .29 to .69), pictures ICC = .86 (95% CI, .79 to .91), colors ICC = .73 (95% CI, .59 to .82), and total score ICC = .86 (95% CI, .79 to .91).

Conclusion: The high ICC value for total mind map score indicates strong MMAR interrater reliability. Pictures and colors demonstrated moderate to strong interrater reliability. We conclude that the MMAR may be a valid and reliable tool to assess mind maps in medicine. However, further research on the validity and reliability of the MMAR is necessary.

Show MeSH
Related in: MedlinePlus