Quality of The Multiple-Choice Questions of The Written Tests for Endocrinology and Metabolism Specialist Board Exams

10.6145/jme.201906_23(2).0006

Feng-Hsuan Liu；Deng-Huang Su；Wei-Yih Chiu；Wan-Chen Wu；Hsuan-Ju Huang；Hsin-Yu Su；Fen-Yu Tseng

endocrinology and metabolism specialist board exams ； multiple-choice questions ； degree of difficulty ； correct rate ； difficulty index ； discrimination index

Journal of Medical Education

23卷2期（2019 / 06 / 01）

125 - 132

英文

Background: The written test of the Endocrinology and Metabolism specialist board exam (EMSBE) contained 100 multiple-choice questions (MCQs). The objectives of this study were to evaluate and improve the quality of the MCQs. Methods: The written test was formatted by considering the distributions of the degree of difficulty (DD) of each question marked by the expert who posed the question. The MCQs of the years 2016 and 2018 were used to analyze the correct rate (CR), the difficulty index (PI), and the discrimination index (DI) of each item. Associations between DD and CR, PI, or DI were evaluated. Results: The average CR, PI, and DI was 59.9 % ± 24.6 %, 0.58 ± 0.24, and 0.30 ± 0.27, respectively, in 2016 and 65.0% ± 25.8%, 0.60 ± 0.26, and 0.27 ± 0.32, respectively, in 2018. Defined by 0.35 ≤ PI ＜ 0.60, there were 33 items and 24 items with ＂optimal＂ difficulty in 2016 and 2018, respectively. Defined by DI ≥ 0.40, there were 33 items and 35 items with ＂very good＂ discriminatory power in 2016 and 2018, respectively. The associations between DD and CR, PI, or DI were statistically non-significant, both in 2016 and 2018. Conclusions: Not all MCQ items used in the written test for EMSBE had optimal difficulty or sufficient discriminatory power. The DD marked by the expert who posed the question was not associated with CR, PI, or DI. This study reviewed the MCQ items of EMSBE and established the basis for future quality improvements.

1. Abdulghani, HM,Ahmad, F,Irshad, M(2015).Faculty development programs improve the quality of multiple choice questions items’ writing.Sci Rep,5,9556.
2. Abdulghani, HM,Irshad, M,Haque, S(2017).Effectiveness of longitudinal faculty development programs on MCQs items writing skills: a followup study.PLoS One,12,e0185895.
3. Al Muhaissen, SA,Ratka, A,Akour, A(2019).Quantitative analysis of single best answer multiple choice questions in pharmaceutics.Curr Pharm Teach Learn,11,251-257.
4. Coughlin, PA,Featherstone, CR(2017).How to write a high quality multiple choice question (MCQ): a guide for clinicians.Eur J Vasc Endovasc Surg,54,654-658.
5. Fayyaz, Khan H,Farooq, Danish K,Saeed, Awan A(2013).Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions.Pak J Med Sci,29,715-718.
6. Feldman, M,Fernando, O,Wan, M(2018).Testing test-enhanced continuing medical education: A randomized controlled trial.Acad Med,93,S30-S36.
7. Fozzard, N,Pearson, A,du Toit, E(2018).Analysis of MCQ and distractor use in a large first year Health Faculty Foundation Program: assessing the effects of changing from five to four options.BMC Med Educ,18,202.
8. Hingorjo, MR,Jaleel, F(2012).Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency.J Pak Med Assoc,62,142-147.
9. Khafagy, G,Ahmed, M,Saad, N(2016).Stepping up of MCQs' quality through a multi-stage reviewing process.Educ Prim Care,27,299-303.
10. McCoubrie, P(2004).Improving the fairness of multiple-choice questions: a literature review.Med Teach,26,709-712.
11. Onishi, H,Park, YS,Takayanagi, R(2018).Combining scores based on compensatory and noncompensatory scoring rules to assess resident readiness for unsupervised practice: implications from a national primary care certification examination in Japan.Acad Med,93,S45-S51.
12. Palmer, E,Devitt, P(2006).Constructing multiple choice questions as a method for learning.Ann Acad Med Singapore,35,604-608.
13. Pham, H,Trigg, M,Wu, S(2018).Choosing medical assessments: Does the multiple-choice question make the grade?.Educ Health,31,65-71.
14. Raymond, MR,Stevens, C,Bucak, SD(2019).The optimal number of options for multiple-choice questions on high-stakes tests: application of a revised index for detecting nonfunctional distractors.Adv Health Sci Educ Theory Pract,24,141-150.
15. Rush, BR,Rankin, DC,White, BJ(2016).The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value.BMC Med Educ,16,250.
16. Scott, KR,King, AM,Estes, MK(2019).Evaluation of an intervention to improve quality of singlebest answer multiple-choice questions.West J Emerg Med,20,11-14.
17. Sim, SM,Rasiah, RI(2006).Relationship between item difficulty and discrimination indices in true/false type multiple choice questions of a para-clinical multidisciplinary paper.Ann Acad Med Singapore,35,67-71.
18. Wakeford, R,Ludka, K,Woolf, K(2018).Fitness to practice sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study.BMC Med,16,230.
19. Webb, EM,Phuong, JS,Naeger, DM(2015).Does educator training or experience affect the quality of multiple-choice questions?.Acad Radiol,22,1317-1322.
20. Yu, Tsao Pan P,Wang, Chia Wei W,Loh, Poey, L(2016).MCQ-construction improves quality of essay assessment among undergraduate dental students.Singapore Dent J,37,37-40.
21. Zahra, D,Burr, SA(2018).Analysis of question text properties for equality monitoring.Perspect Med Educ.,7,401-407.