多層面Rasch模式在數學實作評量的應用

An Application of Many-Facet Rasch Model to Evaluate Mathematics Performance Assessment

10.6251/BEP.20121101.1

謝如山(Ju-Shan Hsieh)；謝名娟(Ming-Chuan Hsieh)

多層面Rasch模式 ； 評分者信度 ； 實作評量 ； many-facet Rasch ； performance assessment ； rater reliability

教育心理學報

45卷1期（2013 / 09 / 01）

1 - 18

繁體中文

本研究採用多層面的Rasch分析，來評估學生在實作評量所具備的潛在能力。本研究所使用的測驗內容在測量學生對數學的數感及生活應用的數學能力，總共有四個考題，三百一十四位四到六年級的學童，與三位評分者參與本研究。結果顯示即使評分者接受訓練並依據規準來進行評分，不同評分者之間的嚴厲度也確實存在差異，透過多層面的Rasch模式分析，可以將評分嚴厲度加以考量，並能協助研究者，偵測評分者不合常規的評分現象，而數據中所呈現之標準化殘差值大致為隨機分布，顯示評分並無明顯的系統性誤差存在。由本研究結果可看出多層面Rasch模式在分析實作評量上相當有用，值得其他教育工作者參考。

The purpose of this study is to evaluate student's potential ability on mathematics ability using Many-facet Rasch Model. The test students took consisted of four ranking levels. Three hundred and fourteen elementary students, and three raters were participated in this study. The results show that even with training and delineating a standard for grading, there remained differences in grader severity among the raters. Many-facet Rasch analysis enabled calibration of grader severity for researchers to detect grader irregularities. Data showed that the standard residuals were randomly distributed, indicating that there were no obvious systematic errors. Overall, the study show Many-facet Rasch model can be quite a useful analytic tool in performance assessment.

1. 姚漢禱、姚偉哲(2007)。應用多層面Rasch 模式分析雙不定向飛靶優秀選手的射擊技術。測驗學刊，55(1)，89-104。
   連結：
2. 張新立、吳舜丞(2008)。多層面Rasch 模式於學術研討會論文評分之應用。測驗學刊，55(1)，105-128。
   連結：
3. Basturk, R.(2008).Applying the many-facet Rasch model to evaluate powerpoint presentation performance in higher education.Assessment & Evaluation in Higher Education,33(4),431-444.
4. de Ayala, R. J.(2009).The theory and practice of item response theory.New York, NY:Guilford.
5. Engelhard, G. J.(1992).The measurement of writing ability with a many-faceted Rasch model.Applied Measurement in Education,5,171-191.
6. Gardner, H.(1993).Frames of mind: The theory of multiple intelligences.New York, NY:Basic Books.
7. Jurdak, M.,Zein, R. A.(1998).The effect of journal writing on achievement in and attitudes toward mathematics.School Science & Mathematics,98(8),412-419.
8. Linacre, J. M.(1999).Investigating rating scale category utility.Journal of Outcome Measurement,3,103-122.
9. Linacre, J. M.(2006).Winsteps: Rasch model statistical software.Chicago, IL:MESA.
10. Linacre, J. M.(2006).FACETS: Many-facet Rasch measurement computer program.Chicago, IL:MESA.
11. Linacre, J. M.(1989).Many-facet Rasch measurement.Chicago, IL:MESA Press.
12. Lunz, M. E.,Stahl, J. A.(1990).Judge consistency and severity across grading periods.Evaluation & the Health Professions,13,435-444.
13. Lunz, M. E.,Wright, B. D.,Linacre, J. M.(1990).Measuring the impact of judge seeerity on examination scores.Applied Measurement in Education,3,331-345.
14. Lunz, M. E.,Wright, B. D.,Stahl, J. A.,Linacre, J. M.(1989).Equating practical examinations.annual meeting of the National Council on Measurement in Education,San Francisco, CA.:
15. Shavelson, R. J.,Baxter, G. P.,Pine, J.(1992).Performance assessment: Political rhetoric and measurement reality.Educational Researcher,21(4),22-27.
16. Smith, E. V.,Kulikowich, J. M.(2004).An application of generalizability theory and many facet Rasch measurement using a complex problem solving skills assessment.Educational and Psychological Measurement,64(4),617-639.
17. Stiggins, R. J.(1994).Student-centered classroom assessment.New York, NY:Macmillan.
18. Tennant, A.,Pallant, J.(2006).Unidimensionality matters! (A tale of two Smiths?).Rasch Measurement Transactions,20(1),1048-1051.
19. Twing, J.,Williams, K. T.(1992).An investigation of writing assessment using a many-faceted Rasch model.annual meeting of the American Educational Research Association,San Francisco, CA.:
20. 王文中(2004)。Rasch 測量模式與其在教育與心理之應用。教育與心理研究，27(4)，637-694。
21. 余民寧(2009)。試題反應理論及其應用。台北=Taipei:心理=Psychological Publishing。
22. 張麗麗(2002)。從分數的意義談實作評量效度的建立。教育研究月刊，98，37-50。
23. 曾安如(2004)。國立台中師範學院教育測驗統計研究所=National Taichung University of Education。
24. 詹元智(2002)。屏東師範學院教育心理與輔導研究所=National Pingtung University of Education。
25. 蔡正濱(2006)。國立屏東教育大學教育心理與輔導學系=National Pingtung University of Education。
26. 藍珮君(2012)。以多面向Rasch 測量模式分析TOCFL 口語測驗評分者訓練效果。永續教育發展－創新與實踐論文集：2010 年國際學術研討會－測驗及評量論文專輯，新北市=New Taipei City:

1. 許婉儀,張惠環,何德華(2023)。對話者之語言能力與評分嚴苛度對印尼語口語評量成績之影響。教育心理學報，55(1)，25-46。
2. 謝名娟(2017)。誰是好的演講者？以多層面Rasch 來分析校長三分鐘即席演講的能力。教育心理學報，48(4)，551-566。
3. 謝名娟(2020)。從多層面Rasch模式來檢視不同的評分者等化連結設計對參數估計的影響。教育心理學報，52(2)，415-436。
4. 張淑華,林巾凱,吳慧珉(2021)。納入評分者嚴苛度之幼兒姿勢動作分析。測驗學刊，68(4)，263-285。
5. 鄭英耀、何曉琪、王佳琪(2016)。科學想像力圖形測驗之發展。教育科學研究期刊，61(4)，177-204。