以支援向量機處理題型符號與文字特徵應用於微積分試題難度分類

Difficulty Level Classification of Calculus Exam Questions Using SVMs with Descriptive Features of Symbols and Texts

林泓宏(Horng-Horng Lin)；蘇家鈜(Chia-Hong Su)；張勝麟(Shing-Lin Chang)

支援向量機 ； 文字特徵 ； 微積分 ； 試題難度分類 ； 題型符號 ； calculus ； question description symbol ； question difficulty level classification ； support vector machine ； text features

測驗學刊

68卷2期（2021 / 06 / 30）

75 - 99

繁體中文

本研究主要在建立微積分試題的「題型符號與文字特徵」，透過人工歸納，對各類型的微積分試題擷取試題符號特徵，並轉換為向量表示。接著，對試題特徵向量，分別以主成分分析（Principal Component Analysis,PCA）及線性判別分析（Linear Discriminant Analysis, LDA）做降維處裡，找出較符合試題難易分布的特徵空間，最後利用支援向量機對降維後的試題特徵，估計試題的難易度，透過使用支援向量機RBF核函數進行「難、中、易」之試題分類。就文獻探討所知，本研究所提出的「題型符號與文字特徵」計算表示形式，為國內外相關研究中創新的特徵集設計。實驗結果顯示：在5摺交叉驗證測試下，對單一摺測試集之微積分試題難易度分類，最高可獲取95%的正確率，而5摺的平均測試正確率也可達90.19%，基於實驗測試結果遠高於隨機亂猜的33.33%，而對3個類別中，隨機亂猜的95%信賴區間上限約在42.69%，可看出本研究方法的實驗結果大幅高於亂猜達47.5%，顯示本研究所提出的「題型符號與文字特徵」對於微積分試題難易度分類具有顯著的功效。

A new design of ＂descriptive symbol and text features＂ of calculus exam questions has been proposed in this paper. The proposed descriptive features of symbols and texts can be extracted from various calculus questions and are represented by vectors. The high dimensionality of extracted features from test questions is then reduced by principal component analysis (PCA) or by linear discriminant analysis (LDA) for finding a lower dimensional feature space that better fits the difficulty-level distribution of test questions. Subsequently, a support vector machine with radial basis kernel is adopted to categorize calculus questions into three degrees of difficulty, i.e., hard, medium and easy. To the best of our knowledge, the proposed descriptive feature representation of symbols and texts of mathematical questions is a novel design for difficulty level estimation of calculus exam questions and is rarely seen in previous literature. In our experiments of difficulty level classification with 5-fold cross validation (CV), the highest classification accuracy of difficulty level of calculus questions in a test fold is 95%, while the average classification accuracy of 5-fold CV is 90.19%. These results are far higher than the mere 33.33% accuracy of random guess. For the three categories, the upper limit of the 95% confidence interval for random guess is about 42.69%. It can be seen that our result is much higher than the upper limit of random guess by about 47.5%. Validate the significant effectiveness of the proposed descriptive features of symbols and texts of calculus exam questions on automatic difficulty level prediction.

1. Abdi, H.,Williams, L. J.(2010).Principal component analysis.Wiley Interdisciplinary Reviews: Computational Statistics,2(4),433-459.
2. Alkharusi, H.(2012).Categorical variables in regression analysis: A comparison of dum-my and effect coding.International Journal of Education,4(2),202-210.
3. Bichi, A. A.(2016).Classical test theory: An introduction to linear modeling approach totest and item analysis.International Journal for Social Studies,2(9),27-33.
4. Chang, S.-L.,Cheng, S.-C.(2017).Computer adaptive learning platform for calculus.Emerging technologies for education,New York, NY:
5. Cheng, S.-C.,Chang, S.-L.(2016).Assessment of the difficulty of items in computer adaptive learning platform for calculus.2016 International Symposium on Novel and Sustainable Technology,Tainan, Taiwan:
6. Cohen, J.,Cohen, P.,West, S. G.,Aiken, L. S.(2013).Applied multiple regression/correlation analysis for the behavioral sciences.London, UK:Routledge.
7. Dey, A.,Chowdhury, S.,Ghosh, M.(2017).Face recognition using ensemble support vector machine.2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN),Kolkata, India:
8. Dignath, C.,Büttner, G.(2008).Components of fostering self-regulated learning among students: A meta-analysis on intervention studies at primary and secondary school level.Metacognition and Learning,3(3),231-264.
9. Dignath, C.,Büttner, G.,Langfeldt, H. P.(2008).How can primary school students learn self-regulated learning strategies most effectively? A meta-analysis on self-regulation training programmes.Educational Research Review,3(2),101-129.
10. Fong, C. J.,Krause, J. M.(2014).Lost confidence and potential: A mixed methods study of underachieving college students’ sources of self-efficacy.Social Psychology of Education,17(2),249-268.
11. Foshee, C. M.,Elliott, S. N.,Atkinson, R. K.(2016).Technology-enhanced learning in college mathematics remediation.British Journal of Educational Technology,47(5),893-905.
12. Hattie, J.,Biggs, J.,Purdie, N.(1996).Effects of learning skills interventions on student learning: A meta-analysis.Review of Educational Research,66(2),99-136.
13. Johnson, B. G.,Phillips, F.,Chase, L. G.(2009).An intelligent tutoring system for the accounting cycle: Enhancing textbook homework with artificial intelligence.Journal of Accounting Education,27(1),30-39.
14. Kerr, P.(2016).Adaptive learning.ELT Journal,70(1),88-93.
15. Koedinger, K. R.,Aleven, V.(2007).Exploring the assistance dilemma in experiments with cognitive tutors.Educational Psychology Review,19(3),239-264.
16. Lample, G.,Charton, F.(2020).Deep learning for symbolic mathematics.2020 Eighth International Conference on Learning Representations,Addis Ababa, Ethiopia:
17. Le, Y.,Porwal, A.,Hdden, E. J.,Dentith, M.(2012).Towards automatic lithological classification from remote sensing data using support vector machines.Computers & Geosciences,45,229-239.
18. Leidinger, M.,Perels, F.(2012).Training self-regulated learning in the classroom: Development and evaluation of learning materials to train self-regulated learning during regular mathematics lessons at primary school.Education Research International,1-14.
19. Lin, C.-C.,Guo, K.-H.,Lin, Y.-C.(2016).A simple and effective remedial learning system with a fuzzy expert system.Journal of Computer Assisted Learning,32(6),647-662.
20. Lord, F. M.(1980).Applications of item response theory to practical testing problems.Hillsdale, NJ:Lawrence Erlbaum Associates.
21. Mok, M. M. C.,Cheng, Y. C.,Moore, P. J.,Kennedy, K. J.(2006).The development and validation of the Self-Learning Scales (SLS).Journal of Applied Measurement,7(4),418-449.
22. Müller-Putz, G.,Scherer, R.,Brunner, C.,Leeb, R.,Pfurtscheller, G.(2008).A closer look on BCI results.International Journal of Bioelektromagnetism,10,52-55.
23. Pinar, R.,Oz, H.(2011).Validity and reliability of the Philadelphia Geriatric Center Morale Scale among Turkish elderly people.Quality of Life Research,20,9-18.
24. Putel, N. J.,Jlavier, R. H.(2015).Detecting packet dropping misbehaving nodes using support vector machine (SVM) in MANET.International Journal of Computer Applications,122(4),26-32.
25. Rao, R. C. (1948). The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society, Series B, 10(2), 159-203.
26. Rasch, G.(1980).Probabilistic models for some intelligence and attainment tests.Chicago, IL:The University of Chicago Press.
27. Walkington, C. A.(2013).Using adaptive learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes.Journal of Educational Psychology,105(4),932-945.
28. Xu, X.,Douglas, J.(2006).Computerized adaptive testing under nonparametric IRT models.Psychometrika,71,121-137.
29. Yarnall, L.,Means, B.,Wetzel, T.(2016).Lessons learned from early implementations of adaptive courseware.Menlo Park, CA:SRI International.
30. 余民寧(2009).試題反應理論（IRT）及其應用.臺北市:心理.
31. 楊智為(2007)。臺中市，國立臺中教育大學。