|
|||||
|
|
Identification-Based Multiple-Choice Assessments in Anatomy can be as Reliable and Challenging as Their Free-Response Equivalents |
|
| Authors: | Jan Douglas-Morris Helen Ritchie Catherine Willis Darren Reed |
| Institution: | 1. School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia
Contribution: Conceptualization (lead), Data curation (equal), Formal analysis (equal), Methodology (equal), Project administration (equal), Resources (equal), Writing - original draft (equal), Writing - review & editing (equal);2. School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia Contribution: Data curation (equal), Formal analysis (equal), Methodology (equal), Writing - original draft (equal), Writing - review & editing (equal);3. School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia Contribution: Conceptualization (equal), ?Investigation (equal), Methodology (equal), Project administration (equal), Writing - review & editing (equal);4. School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia |
| Abstract: | Multiple-choice (MC) anatomy “spot-tests” (identification-based assessments on tagged cadaveric specimens) offer a practical alternative to traditional free-response (FR) spot-tests. Conversion of the two spot-tests in an upper limb musculoskeletal anatomy unit of study from FR to a novel MC format, where one of five tagged structures on a specimen was the answer to each question, provided a unique opportunity to assess the comparative validity and reliability of FR- and MC-formatted spot-tests and the impact on student performance following the change of test format to MC. Three successive year cohorts of health science students (n = 1,442) were each assessed by spot-tests formatted as FR (first cohort) or MC (following two cohorts). Comparative question difficulty was assessed independently by three examiners. There were more higher-order cognitive skill questions and more of the course objectives tested in the MC-formatted tests. Spot-test reliability was maintained with Cronbach’s alpha reliability coefficients ≥ 0.80 and 80% of the MC items of high quality (having point-biserial correlation coefficients > 0.25). These results also demonstrated guessing was not an issue. The mean final score for the MC-formatted cohorts increased by 4.9%, but did not change for the final theory examination that was common to all three cohorts. Subgroup analysis revealed that the greatest change in spot-test marks was for the lower-performing students. In conclusion, our results indicate spot-tests formatted as MC are suitable alternatives to FR tests. The increase in mean scores for the MC-formatted spot-tests was attributed to the lower demand of the MC format. |
| Keywords: | gross anatomy education musculoskeletal anatomy practical assessment selected-response item Bloom’s taxonomy test utility |