自然科成就測驗試題難度成份分析

The Cognitive Components of Item Difficulty of a Science Achievement Test

10.6479/JE.201211.0155

胡智超(Chih-Chao Hu)；凃柏原(Bor-Yaun Twu)

認知成份分析 ； 類別迴歸 ； 線性羅吉斯潛在特質模式 ； cognitive components analysis ； categorical regression ； linear logistic latent trait model

教育學誌

28期（2012 / 11 / 01）

155 - 191

繁體中文

本研究使用認知成份分析的觀點，利用線性羅吉斯潛在特質模式（LLTM）和類別迴歸分析兩種方法來探討試題的難度成份與由Rasch模式所估計得到的難度參數之間的關係，並比較由這兩個分析方法所得到試題難度有效的預測變項（難度成份）之異同。研究所用自然科試題取自於國內某一個成就測驗題庫建置計畫中五年級的部分，一共有176個試題，由三個年度的資料所組成，總共有4918名受試者作答。研究中所指稱之難度成份包括：（1）文本特徵（例如：圖表的有無…等）；（2）認知特徵，亦即包含概念理解、科學探究和實務推理三個層次的認知水準；（3）知識特徵，亦即試題所屬的內容領域。研究結果主要如下：一、類別迴歸分析與LLTM分析所得之預測變項不盡相同，類別迴歸中僅有知識特徵，而LLTM中則是所有難度成份都有顯著。二、類別迴歸分析與LLTM分析的預測難度相關在-.645～-.713之間，兩者預測難度與Rasch難度的相關分別為.370～.471與-.544～-.618之間。

The purpose of this research was to study the relationship between the cognitive components of item difficulty and the item difficulty parameter estimated by using the Rasch model based on the view of cognitive component analysis. The Latent logistic latent trait model (LLTM) and categorical regression were used to find out the efficient predictors (cognitive components) and the results given by the two methods were compared. The empirical data from the science assessment of an item pool was used. The sample consisted of 4,918 fifth graders from the Southern Taiwan and 176 items from the 2005, 2006 and 2008 administrations.The sources of processing difficulty, identified using cognitive component analysis, include (1) text attribute (e.g., presence of a figure, etc.); (2) cognitive level, included conceptual understanding, scientific investigation, and practical reasoning; (3) knowledge characteristics, i.e., content fields of items. The relationship between Rasch difficulty parameter and different sources of processing difficulty was investigated by using the categorical regression and the logistic latent trait model (LLTM).The main findings of this study were as follows:1. There are differences between the cognitive components identified by the categorical regression analysis and the LLTM. In the categorical regression analysis, item content is the only significant component, but in the LLTM analysis, all cognitive components are significant.2. The correlation between the difficulties predicted by the categorical regression analysis and the LLTM is between -.645 and -.713. The correlation between the difficulties predicted by the categorical regression analysis and the Rasch difficulty parameter is between .370 and .471, and the correlation between the difficulty predicted by the LLTM and the Rasch difficulty is between -.544 and -.618.

1. Sternberg, Robert J.、Williams, Wendy Melissa、周甘逢譯、劉冠麟譯(2002)。教育心理學。臺北市:華騰文化。
   連結：
2. 張銘秋、謝秀月、徐秋月(2009)。PISA科學素養之試題認知成份分析。課程與教學季刊，13(1)，1-19。
   連結：
3. Dimitrov, D. M.,Raykav, T.(2003).Validation of cognitive structures : A structural equation modeling approach.Multivariate Behavioral Research,38(1),1-23.
4. Embreston, S. E.,Gorin, J.(2001).Improving construct validity with cognitive psychology principles.Journal of Education Measurement,38,343-368.
5. Emmerich, W.(1989).,Princeton, NJ:Educational Testing Service.
6. Enright, M. K.,Allen, N.,Kim, M.(1993).,Princeton, NJ:Educational Testing Service.
7. Fischer, G. H.(1973).The linear logistic test model as an instrument in educational research.Acta Psychogica,37,359-374.
8. Fischer, G. H.(1995).The liner logistic model.Rasch models: Foundations, recent development, and application,New York:
9. Frederiksen, N.(ed.),Mislevy, R. J.(ed.),Bejar, I. I.(ed.)(1993).Test theory for a new generation of test.Hillsdale, New Jersey:Lawrence Erlbaum Association.
10. Gorin, J. S.(2005).Manipulating processing difficult of reading comprehension questions: The feasibility of verbal item generation.Journal of Educational Measurement,42(4),351-373.
11. Gorin, J. S.,Embretson, S. E.(2006).Item difficult modeling of paragraph comprehension items.Applied Psychological Measurement,30(5),394-411.
12. Green, K. E.,Smith, R. M.(1987).A comparison of two methods of decomposing item difficulties.Journal of Educational Statistic,12(4),369-381.
13. Mayer, Richard E.、林清山譯(2002)。教育心理學：認知取向。臺北:遠流。
14. Pellegrino, J. W.(ed.),Chudowsky, N.(ed.),Glaser, R.(ed.)(2001).Knowing what students know: The science and design of educational assessment.Washington, DC:National Research Council.
15. Rosca, C. V.(2004).Boston,Boston College.
16. Sternberg, R. J.(1978).,Washington, DC:Office of Naval Research.
17. Yepes-Baray, M.(1997).Lessons learned form the coding of item attributes for the 1996 NAEP science assessment G4.Annual Meeting of the National Council in Measurement in Education,Chicago, IL:
18. Yepes-Baray, M.(1996).A cognitive study based on the NAEP Science Assessment.Annual Meeting of the National Council on Measurement in Education,New York, NY:
19. 丁振豐(1995)。幾何圖形類比推理題目內容成份分析之研究。臺南師院學報，28，83-114。
20. 李岳勳(2004)。碩士論文(碩士論文)。臺南市，國立臺南師範學院。
21. 林世華、葉嘉惠(1999)。數字系列完成測驗試題認知成份分析之研究。教育心理學報，31(1)，139-165。
22. 洪碧霞、林素微、林娟如(2006)。認知複雜度分析架構對TASA-MAT六年級線上測驗試題難度的解釋力。教育研究與發展期刊，2(4)，69-86。
23. 張春興(2002)。教育心理學─三化取向的理論與實踐。臺北市:東華書局。
24. 郭生玉(2003)。心理與教育測驗。臺北市:精華書局。
25. 歐滄和(2002)。教育測驗與評量。臺北市:心理出版社。
26. 盧思丞、凃柏原(2012)。TIMSS八年級與國中基測物理試題認知成份之探討。永續教育發展─創新與實踐論文集，新北市:
27. 凃柏原、梁恩琪、翁大德、楊毅立(2004)。國中基測自然科試題分析研究。科技化測驗與能力指標評量國際研討會