Improving Knowledge Retention in a Systems Programming Course Through Confirmatory Multiple-Choice Questions

應用信心指標選擇題以保留學習知識：以系統程式課程為例

10.6925/SCJ.202203_18(1).0004

賴建宏(Chien-Hung Lai)；鍾斌賢(Bin-Shyan Jong)；夏延德(Yen-Teh Hsia)；林聰武(Tsong-Wuu Lin)

ultiple choice ； knowledge retention ； assessment ； evaluation methodology ； learning strategy ； 選擇題 ； 知識保留 ； 測驗評估 ； 評估方法 ； 學習策略

教育研究與發展期刊

18卷1期（2022 / 03 / 31）

107 - 127

英文

Multiple-choice questions (MCQs) are often used in assessments. Although MCQs can be designed to test higher-order cognitive skills such as analysis, synthesis, and evaluation, they are often used to measure recall. This may encourage students to only learn material superficially. To resolve this problem, several variations of MCQs have been proposed to improve learning. We proposed a modified type of MCQ called a ＂confirmatory MCQ＂ (CMCQ). Optional online CMCQ tests were more effective than MCQs and confidence-based marking tests in helping students study material from a systems programming course because they prompted students to thoroughly evaluate each answer.

選擇題通常用於評估之目的。儘管根據先前研究選擇題被設計為可測試及評估更高階思維之認知處理，例如分析、綜合和評估，但多數時候選擇題仍然被使用於回測對於知識的短暫記憶保留，導致學習者為了達到良好的學習成就而採取死記硬背的方式，無法確實將知識長期記憶。有鑑於選擇題所擁有的潛在缺點，因此不少研究提出了改善選擇題的方式，而這些研究皆是為了讓學習者進行選擇題測驗，於進行形成性評估時能有更好的學習效果。本研究中提出了另一種的選擇題模式，稱為信心指標選擇題（confirmatory multiple-choice question, CMCQ），即測驗過程需給予每一選項信心指標，以表示對於該選項的確認程度，因此學生面對每一選項皆要全盤思考，將腦中學習過的課程知識整合，方能作出正確答案。同時為了檢測本研究設計能有效達到知識保留，讓學生在作答過程可以回顧先前學習過的內容，因此與一般的選擇題測驗及僅給予作答結果信心指標的選擇題測驗（confidence-based marking multiple-choice question, CBM-Based MCQ）進行比較，實驗結果顯示使用信心指標的選擇題測驗複習課程內容可以顯著幫助學習者更有效記住課程內容，這也是因為相較於回答一般選擇題或CBM-Based MCQ相比，對於每個選項都要確認信心指標的CMCQ更能刺激學生進行更徹底的思考，對學習過的內容反思，以達到知識保留。

1. Ang, K. C., & Boo, H. K. (2006, November 22-24). Exploring the use of reflective MCQ in primary science formative assessment [Paper presentation]. 2006 International Science Education Conference, Singapore. https://repository.nie.edu.sg/bitstream/10497/14904/3/ISEC-2006-86_a.pdf.
2. Candel, C., Vidal-Abarca, E., Cerdán, R., Lippmann, M., & Narciss, S. (2020). Effects of timing of formative feedback in computer-assisted learning environments. Journal of Computer Assisted Learning, 36(5), 718-728. https://www.doi.org/10.1111/jcal.12439。
3. Fies, C., & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101-109. https://doi.org/10.1007/s10956-006-0360-1。
4. Holley, C. D., & Dansereau, D. F. (2014). Spatial learning strategies: Techniques, applications, and related issues. Academic Press.
5. Hsia, Y. T., Jong, B. S., Lin, T. W., & Liao, J. Y. (2019). Designating “hot” items in multiplechoice questions—A strategy for reviewing course materials. Journal of Computer Assisted Learning, 35(2), 188-196. https://doi.org/10.1111/jcal.12320。
6. Huang, J., Kerdphol, S., & Inthong, W. (2018). Comparing the effectiveness of rote learning and meaningful learning on English vocabulary memorization for 7th grade students at padoongrasdra school. Humanities and Social Sciences Journal of Graduate School, Pibulsongkram Rajabhat University, 13(1), 295-304.
7. Lai, C. H., Lee, T. P., Jong, B. S., & Hsia Y. T. (2014). Using SPRT+ to Reduce Measure Time on Student Learning Efficiency by Pre-defined Student’s Confidence Indicator. International Journal of Emerging Technologies in Learning, 9(3), 55-58. https://doi.org/10.3991/ijet.v9i3.3317。
8. Lai, C. H., Jong, B. S., Hsia, Y. T., & Lin, T. W. (2021). Association questions on knowledge retention. Educational Assessment, Evaluation and Accountability, 33(2), 375-390. https://doi.org/10.1007/s11092-020-09337-5。
9. Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., Bulger, M., Campbell, J., Knight, A., & Zhang, H. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary educational psychology, 34(1), 51-57. https://doi.org/10.1016/j.cedpsych.2008.04.002。
10. Novak, J. D. (1990). Concept mapping: A useful tool for science education. Journal of research in science teaching, 27(10), 937-949. https://doi.org/10.1002/tea.3660271003。
11. Pamphlett, R., & Farnill, D. (1995). Effect of anxiety on performance in multiple choice examination. Medical education, 29(4), 297-302. https://doi.org/10.1111/j.1365-2923.1995.tb02852.x。
12. Paxton, M. (2000). A linguistic perspective on multiple choice questioning. Assessment & Evaluation in Higher Education, 25(2), 109-119. https://doi.org/10.1080/713611429。
13. Ruiz-Primo, M. A., Shavelson, R. J., Li, M., & Schultz, S. E. (2001a). On the validity of cognitive interpretations of scores from alternative concept-mapping techniques. Educational Assessment, 7(2), 99-141. https://doi.org/10.1207/S15326977EA0702_2。
14. Ruiz-Primo, M. A., Schultz, S. E., Li, M., & Shavelson, R. J. (2001b). Comparison of the reliability and validity of scores from two concept-mapping techniques. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 38(2), 260-278. https://doi.org/10.1002/1098-2736 (200102)38:23.0.CO;2-F。
15. Sas, I. C. (2010). The Multiple-choice concept map (MCCM): An interactive computer-based assessment method. University of Nevada, Las Vegas.
16. Schau, C., Mattern, N., Zeilik, M., Teague, K. W., & Weber, R. J. (2001). Select-and-fill-in concept map scores as a measure of students’ connected understanding of science. Educational and Psychological Measurement, 61(1), 136-158. https://doi.org/10.1177/00131640121971112。
17. Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4), 453-472. https://doi.org/10.1023/A:1003196224280。
18. Snyder, L. G., & Snyder, M. J. (2008). Teaching critical thinking and problem solving skills. The Journal of Research in Business Education, 50(2), 90-99.
19. Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today, 26(8), 662-671. https://doi.org/10.1016/j.nedt.2006.07.006。
20. Tian, Z., Zhang, K., Zhang, T., Dai X. & Lin, J. (2019). Application of Ausubel cognitive assimilation theory in teaching/learning medical biochemistry and molecular biology. Biochemistry and Molecular Biology Education, 48(3), 202-219. https://doi.org/10.1002/bmb.21327。
21. Tsai, F. H., Tsai, C. C., & Lin, K. Y. (2015). The evaluation of different gaming modes and feedback types on game-based formative assessment in an online learning environment. Computers & Education, 81, 259-269. https://doi.org/10.1016/j.compedu.2014.10.013。