利用log文檔探析PISA 2012學生解難模式及相關認知和心理因素

Using Log File Analyses to Examine PISA 2012 Student Problem-Solving Patterns and the Associated Cognitive and Psychological Factors

金松麗(Song-Li Jin)；張國祥(Kwok-Cheung Cheung)；薛寶嫦(Pou-Seong Sit)

log文檔 ； PISA 2012電子測試 ； 表現指標 ； 教育數據挖掘 ； 解難能力 ； behavioral indicators ； educational data mining ； log file ； PISA 2012 digital test ； problem solving

測驗學刊

63卷4期（2016 / 12 / 01）

227 - 252

繁體中文

解難能力是21 世紀人們需要具備的核心特質之一。20 世紀，由於研究方法的限制及缺乏合適的測評工具，較少學者會研究受試者的解難歷程。隨著資訊科技的進步，人們發展出記錄大樣本和大量解難歷程行為的方式log 文檔。本文以PISA 2012 電子解難測試之一道公開試題（機械人吸塵機）的學生log 文檔作為研究數據，探索出三個用於分析的表現指標：學生解難的時間、解難步驟的數目，以及觀看測試素材的次數，以這些指標比較七個高表現地區和三個低表現地區的狀況，發現：(1)低表現地區學生的解難時間比高表現地區長；(2)低表現地區與高表現地區相比，解難步驟較少的學生百分比和觀看素材次數較少的學生百分比均較高。之後，透過決策樹方法分析出五種學生解難模式，並進行跨地區比較和對應分析，發現：(1)高表現地區學生多為「低頻答對」或「高頻答對」型，低表現地區學生多為「低頻答錯」型；(2)學生整體解難水準和對問題解決持開放態度的水準，與這五種解難模式分別在高表現和低表現地區有不同對應關係。最後，本文根據研究結果為教育工作者提供關於解難教學，以及研究者相關後續研究的建議。

Problem solving is one of the core skills of the 21th century. In the 20th century, due to the limitation of research methodology and adequacy of assessment tools, very few studies focused research on the students' problem-solving processes. With the rapid development of computer science and information technology, log files have been deployed to document students' behavioral processes in large-scale sample surveys. This research studied the student log files of a released item (ROBOT CLEANER) in the PISA 2012 digital problem-solving assessment. Three behavioral indicators had been developed for analyses and comparison between the seven high-performing and the three low-performing economies: response time, number of problem-solving steps, and number of times watching the simulation. The comparison results showed that: (1) the overall response time of students from the low-performing economies was longer than students from the high-performing economies; (2) Comparing between the low- and high-performing economies, the proportion of students of low-performing economies who had fewer number of problem-solving steps was higher. The same was found regarding the number of times watching the simulation. Through the use of decision tree, five problem- solving patterns were uncovered. The correspondence analysis results showed: (1) ＂Low-frequency Answer-correct＂ and ＂High-frequency Answer-correct＂ were the two dominant problem-solving patterns found in the seven high-performing economies, whereas it was ＂Low-frequency Answer-wrong＂ found in low-performing economies; (2) students' levels of overall problem-solving performance and openness to problem solving were having corresponding relationships with the five problem-solving patterns. This research provided recommendations for improving problem-based teaching and learning, and suggestions for pursuing follow-up studies.

1. Dewey, J. (1910). How we think. New York, NY: Heat.
2. Dewey, J. (1916). Method in science teaching. General Science Quarterly, 1(1), 3-9.
3. Beck, J. E.(2004).Using response times to model student disengagement.Proceedings of the ITS 2004 Workshop on Social and Emotional Intelligence in Learning Environments,Berlin, Germany:
4. Bruner, J. S.(1961).The act of discovery.Harvard Educational Review,31,21-32.
5. De Mauro, A.,Greco, M.,Grimaldi, M.(2015).What is big data? A consensual definition and a review of key research topics.AIP Conference Proceedings,USA:
6. Dodonova, Y. A.,Dodonov, Y. S.(2013).Faster on easy items, more accurate on difficult ones: Cognitive ability and performance on a task of varying difficulty.Intelligence,41(1),1-10.
7. Ericsson, K. A.,Simon, H. A.(1980).Verbal reports as data.Psychological Review,87(3),215-251.
8. Fonteyn, M. E.,Kuipers, B.,Grobe, S. J.(1993).A description of think aloud method and protocol analysis.Qualitative Health Research,3(4),430-441.
9. Gick, M. L.,Holyoak, K. J.(1980).Analogical problem solving.Cognitive Psychology,12(3),306-355.
10. Goldhammer, F.,Naumann, J.,Stelter, A.,Tóth, K.,Rölke, H.,Klieme, E.(2014).The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment.Journal of Educational Psychology,106(3),608-626.
11. Goos, M.,Galbraith, P.(1996).Do it this way! Metacognitive strategies in collaborative mathematical problem solving.Educational Studies in Mathematics,30(3),229-260.
12. Greiff, S.,Wüstenberg, S.,Avvisati, F.(2015).Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving.Computers & Education,91,92-105.
13. Greiff, S.,Wüstenberg, S.,Holt, D. V.,Goldhammer, F.,Funke, J.(2013).Computer-based assessment of complex problem solving: Concept, implementation, and application.Educational Technology Research and Development,61(3),407-421.
14. Hershkovitz, A.,Nachmias, R.(2009).Learning about online learning processes and students' motivation through web usage mining.Interdisciplinary Journal of E-Learning and Learning Objects,5(1),197-214.
15. Jonassen, D. H.(2011).Learning to solve problems: A handbook for designing problemsolving learning environments.New York, NY:Routledge.
16. Klösgen, W.,Zytkow, J.(2002).Handbook of data mining and knowledge discovery.New York, NY:Oxford University Press.
17. Lee, Y. J.(2015).Analyzing log files to predict students' problem solving performance in a computer-based physics tutor.Journal of Educational Technology & Society,18(2),225-236.
18. McGaw, B.(Ed.),Peterson, P.(Ed.),Baker, E.(Ed.)(2010).International encyclopedia of education.Oxford, UK:Elsevier.
19. Montague, M.,Applegate, B.(1993).Middle school students' mathematical problem solving: An analysis of think-aloud protocols.Learning Disability Quarterly,16(1),19-32.
20. Organisation for Economic Co-operation and Development=OECD(2014).PISA 2012 results: Creative problem solving: Students' skills in tackling real-life problems.Paris, France:Author.
21. Organisation for Economic Co-operation and Development=OECD(2013).PISA 2012 results: Ready to learn: Students' engagement, drive and self-beliefs.Paris, France:Author.
22. Organisation for Economic Co-operation and Development=OECD(2014).PISA 2012 technical report.Paris, France:Author.
23. Pargament, K. I.,Kennell, J.,Hathaway, W.,Grevengoed, N.,Newman, J.,Jones, W.(1988).Religion and the problem-solving process: Three styles of coping.Journal for the Scientific Study of Religion,27(1),90-104.
24. Peña-Ayala, A.(2014).Educational data mining: A survey and a data mining-based analysis of recent works.Expert Systems with Applications,41(4),1432-1462.
25. Peña-Ayala, A.(Ed.)(2014).Educational data mining.Switzerland:Springer International Publishing.
26. Polya, G.(2014).How to solve it: A new aspect of mathematical method.NJ:Princeton University Press.
27. Romero, C.,Ventura, S.(2007).Educational data mining: A survey from 1995 to 2005.Expert Systems with Applications,33(1),135-146.
28. Romero, C.,Ventura, S.(2010).Educational data mining: A review of the state of the art.Systems, Man, and Cybernetics, Part C Applications and Reviews, IEEE Transactions on,40(6),601-618.
29. Rosenzweig, C.,Krawec, J.,Montague, M.(2011).Metacognitive strategy use of eighthgrade students with and without learning disabilities during mathematical problem solving a think-aloud analysis.Journal of Learning Disabilities,44(6),508-520.
30. Sachin, R. B.,Vijay, M. S.(2012).A survey and future vision of data mining in educational field.Advanced Computing & Communication Technologies (ACCT), 2012 Second International Conference
31. Scherer, R.,Gustafsson, J. E.(2015).The relations among openness, perseverance, and performance in creative problem solving: A substantive-methodological approach.Thinking Skills and Creativity,18,4-17.
32. Sit, P. S.,Cheung, K. C.,Cheong,W. C.,Mak, S. K.(2014).The effects of webpage navigation on digital reading performance.The International Journal of Assessment and Evaluation,20(3),22-36.
33. Srivastava, J.,Cooley, R.,Deshpande, M.,Tan, P. N.(2000).Web usage mining: Discovery and applications of usage patterns from web data.ACM SIGKDD Explorations Newsletter,1(2),12-23.
34. Sternberg, R. J.(1985).Beyond IQ: A triarchic theory of human intelligence.New York, NY:Cambridge University Press.
35. Sweller, J.(1988).Cognitive load during problem solving: Effects on learning.Cognitive Science,12(2),257-285.
36. Trilling, B.,Fadel, C.(2009).21st century skills: Learning for life in our times.San Francisco, CA:Jossey-Bass.
37. Tschirgi, J. E.(1980).Sensible reasoning: A hypothesis about hypotheses.Child Development,51(1),1-10.
38. Wang, M. T.,Eccles, J. S.(2013).School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective.Learning and Instruction,28,12-23.
39. 毛國君、段立娟、王實、石雲(2005)。資料採擷原理與演算法。中國北京市:清華大學出版社。
40. 邱皓政(2015)。從小數據邁向大數據：心理計量學在資料科學中的角色。2015 全球華人教育資訊與評估學術研討會暨中國測驗學會年會，臺中市:
41. 陳雅慧(2015)。未來人才，必備4大關鍵能力。親子天下，66，158-161。
42. 蔣芝芳、趙建康(2014)。滲透「三學」，提升數學學習力。新教育，2014(4)，82-83。

1. 盧秀琴,莊淑芬(2021)。學生及學校因素對合作式問題解決表現之多層級分析。測驗學刊，68(3)，175-207。