LAUSR

PURPOSE Newer electronic differential diagnosis supports (EDS) are efficient and effective at improving diagnostic skill. While these supports are encouraged in practice, they are prohibited in medical licensing examinations. The purpose of this study is to determine how using an electronic differential diagnosis support (EDS) impacts examinees' results when answering clinical diagnosis questions. METHOD The authors recruited 100 medical students from McMaster University (Hamilton, Ontario) to answer 40 clinical diagnosis questions in a simulated examination in 2021. Of these, 50 were first year students and 50 were final year students. Participants from each year of study were randomized to one of two groups. During the survey, half of the students had access to Isabel (an EDS) and half did not. Differences were explored using ANOVA and reliability estimates were compared for each group. RESULTS Test scores were higher for final year versus first year students (53 13% versus 2910, p<0.001) and higher with the use of EDS (44 28% versus 36 26%, p<0.001). Students using the EDS took longer to complete the test (p<.001). Internal consistency reliability (Cronbach's alpha) increased with EDS use among final year students, but was reduced among first year students, although the effect was not significant. A similar pattern was noted in item discrimination, which was significant. CONCLUSION EDS use during diagnostic licensing style questions was associated with modest improvements in performance, increased discrimination in senior students and increased testing time. Given that clinicians have access to EDS in routine clinical practice, allowing EDS use for diagnostic questions would maintain ecological validity of testing while preserving important psychometric test characteristics.