適用於小班教學現場之部分連結神經網路認知診斷模式

A Cognitive Diagnosis Model Based on Partially Connected Neural Network and Its Application on Small-Class Teaching

李政軒(Cheng-Hsuan Li)；謝佩鈞(Pei-Jyun Hsieh)；劉志勇(Zhi-Young Liu)

DBV ； DINA ； GDINA ； 深度學習 ； 部分連結神經網路 ； 認知診斷模式 ； Cognitive diagnosis models ； DINA ； GDINA ； Partially connected neural network ； deep learning

測驗學刊

67卷2期（2020 / 06 / 01）

145 - 166

繁體中文

認知診斷模式已經被應用到不同學科領域中，以細部了解學生在技能概念的精熟與否。然而，常見的參數型認知診斷模式，如DINA（Deterministic Input, Noisy ＂And＂ Gate）模式和廣義DINA（Generalized DINA, GDINA）模式，都需要一定數量的訓練樣本，才能有好的預測效果，故DINA與GDINA不太適用於目前小班制的教學現場。本研究提出一個部分連結的神經網路模式，不同於傳統的全連結神經網路模式，透過試題與技能對應的Q矩陣，來決定試題與概念之間的連結。也就是說，學生的特定技能概念具備與否，應該只與需要使用到該技能概念的試題對錯有關。最後再利用理想作答反應來訓練網路連結的權重，這種訓練方式不需要使用到學生的真實作答反應，只要在命題時，設計好該試卷的Q矩陣，便可以透過Q矩陣推得的理想作答反應來求得網路連結權重。由模擬資料和「分數乘法」實證資料實驗結果顯示，部分連結神經網路認知診斷模式在小樣本的情況下，分類一致性優於DINA模式與GDINA模式。因此，此模式可以適用於目前小班教學現場，甚至可以搭配網路教學平臺進行個人化診斷評量，即在單一學生情況下，也可以進行概念精熟診斷。

Cognitive diagnosis models (CDMs) classify examinees' mastery skill profiles according to their test performance and Q matrix, the mapping between items and skills. They have been applied to many but different research areas such as language assessment, psychology, and international testing. Moreover, they are also be applied to adaptive learning or personalized learning. DINA (Deterministic Input, Noisy ＂And＂ Gate Model) and its generalized version, GDINA (Generalized DINA) are two well-known models in CDMs. However, both of them need a certain amount of examinees' responses to train and pre-determine appropriate models' parameters. Hence, they are not suitable for small-class teaching. In this study, a partially connected neural network for cognitive diagnostic (PNNCD) was proposed. The connections between nodes in the input layer and nodes in the output layer are determined by the Q matrix. The reason is that only items required specific skill can influence the mastery level of the skill. Moreover, the ideal responses, directly determined by the Q matrix of the test, are used to train the model weights. That is, the proposed model is not trained by examinees' responses, and, hence, it can be directly applied to only one examinee situation. According to experiments on both simulated data sets and a real data set, the proposed PNNCD outperforms than DINA and GDINA in small sample size. Therefore, PNNCD is more appropriate to apply in the small-class teaching.

1. 吳慧珉,鄭俊彥,施淑娟(2015)。認知診斷模式之理論與實務。測驗學刊，62，303-328。
   連結：
2. 李政軒(2016)。無參數加權認知診斷模式。測驗學刊，6(2)，133-151。
   連結：
3. Abadi, M.,Barham, P.,Chen, J.,Chen, Z.,Davis, A.,Dean, J.,Kudlur, M.(2016).Tensorflow: A system for large-scale machine learning.Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16),Savannah, GA:
4. Alfadly, M. M. Y.(2018).Kingdom of Saudi Arabia,King Abdullah University of Science and Technology Thuwal.
5. Bradshaw, L.,Izsák, A.,Templin, J.,Jacobson, E.(2014).Diagnosing teachers’ understanding of rational number: Building a multidimensional test within the diagnostic classification framework.Educational Measurement: Issues and Practice,33(1),2-14.
6. Chen, Y.,Li, X.,Liu, J.,Ying, Z.(2018).Recommendation system for adaptive learning.Applied Psychological Measurement,42(1),24-41.
7. Chiu, C. Y.(2013).Statistical refinement of the Q-matrix in cognitive diagnosis.Applied Psychological Measurement,37(8),598-618.
8. Chiu, C. Y.,Sun, Y.,Bian, Y.(2018).Cognitive diagnosis for small educational programs: The general nonparametric classification method.Psychometrika,83(2),355-375.
9. Chiu, C.-Y.,Douglas, J.(2013).A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns.Journal of Classification,30,225-230.
10. Choi, K. M.,Lee, Y. S.,Park, Y. S.(2015).What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics.Eurasia Journal of Mathematics, Science & Technology Education,11(6),1563-1577.
11. de la Torre, J.(2009).DINA model and parameter estimation: A didactic.Journal of Educational and Behavioral Statistics,34(1),115-130.
12. de la Torre, J.(2009).A cognitive diagnosis model for cognitively based multiple-choice options.Applied Psychological Measurement,33(3),163-183.
13. de la Torre, J.(2011).The generalized DINA model framework.Psychometrika,76(2),179-199.
14. de la Torre, J.,van der Ark, L. A.,Rossi, G.(2015).Analysis of clinical data from cognitive diagnosis modeling framework.Measurement and Evaluation in Counseling and Development,1-16.
15. Huebner, A.(2010).An overview of recent developments in Cognitive Diagnostic Computer Adaptive Assessments.Practical Assessment, Research & Evaluation,15(3),1-7.
16. Hwang, G. J.(2003).A conceptual map model for developing intelligent tutoring systems.Computers & Education,40(3),217-235.
17. Hwang, G. J.,Panjaburee, P.,Triampo, W.,Shih, B. Y.(2013).A group decision approach to developing concept effect models for diagnosing student learning problems.British Journal of Educational Technology,44(3),453-468.
18. Jaeger, J.,Tatsuoka, C.,Berns, S.,Varadi, F.(2006).Distinguishing neurocognitive functions using partially ordered classification models.Schizophrenia Bulletin,32,679-691.
19. Junker, B. W.,Sijtsma, K.(2001).Cognitive assessment models with few assumptions, and connections with nonparametric item response theory.Applied Psychological Measurement,25(3),258-272.
20. Kingma, D. P.,Ba, J.(2014).arXiv preprint arXiv:1412.6980arXiv preprint arXiv:1412.6980,未出版
21. Kuo, B. C.,Chen, C. H.,de la Torre, J.(2018).A cognitive diagnosis model for identifying coexisting skills and misconceptions.Applied psychological measurement,42(3),179-191.
22. Kuo, B. C.,Pai, H. S.,de la Torre, J.(2016).Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction.Applied Psychological Measurement,40(5),315-330.
23. LeCun, Y.,Bengio, Y.,Hinton, G.(2015).Deep learning.Nature,521(7553),436-444.
24. Ma, W., & de la Torre, J. (2016). GDINA: The generalized DINA model framework. R package version 0.13.0. Available online at: http://CRAN.R-project.org/package=GDINA
25. Mesnil, G.,Dauphin, Y.,Glorot, X.,Rifai, S.,Bengio, Y.,Goodfellow, I.,Vincent, P.(2011).Unsupervised and transfer learning challenge: A deep learning approach.Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning workshop-Volume 27
26. Mure an, H.,Oltean, M.(2018).Fruit recognition from images using deep learning.Acta Universitatis Sapientiae, Informatica,10(1),26-42.
27. Panjaburee, P.,Hwang, G. J.,Triampo, W.,Shih, B. Y.(2010).A multi-expert approach for developing testing and diagnostic systems based on the concept-effect model.Computers & Education,55(2),527-540.
28. Philip, N. S.(2010).A learning algorithm based on primary school teaching wisdom.Paladyn,1(3),160-168.
29. Rupp, A. A.,Templin, J.,Henson, R. A.(2010).Diagnostic measurement: Theory, methods, and applications.New York, NY:Guilford Press.
30. Schmidhuber, J.(2015).Deep learning in neural networks: An overview.Neural Networks,61,85-117.
31. Sessoms, J.,Henson, R. A.(2018).Applications of diagnostic classification models: A literature review and critical commentary.Measurement: Interdisciplinary Research and Perspectives,16(1),1-17.
32. Tatsuoka, C.(2002).Data-analytic methods for latent partially ordered classification models.Journal of the Royal Statistical Society Series C (Applied Statistics),51,337-350.
33. Tatsuoka, K. K.(1985).A probabilistic model for diagnosing misconceptions by the pattern classification approach.Journal of Educational and Behavioral Statistics,10(1),55-73.
34. Templin, J. L.,Henson, R. A.(2006).Measurement of psychological disorders using cognitive diagnosis models.Psychological Methods,11,287-305.
35. Templin, J. L.,Hoffman, L.(2013).Obtaining diagnostic classification model estimates using Mplus.Educational Measurement: Issues and Practice,32(2),37-50.
36. 林宏憲(2012)。臺中市，亞洲大學。