以學生評鑑教師量表決定教師的開課或去留可行嗎？混合IRT分析取向

Is Using a SRI to Determine the Fate of Teachers or Commencement of Work Suitable?-A Mixture IRT Analysis

曾明基(Ming-Ci Tseng)；邱皓政(Haw-Jeng Chiou)；張德勝(Te-Sheng Chang)；羅寶鳳(Pao-Feng Lo)

學生評鑑教師教學 ； 混合IRT分析 ； student ratings of instruction ； mixture IRT analysis

教育科學研究期刊

58卷1期（2013 / 03 / 01）

91 - 116

繁體中文

本研究主要探討以學生評鑑教師教學量表決定大學教師開課或去留的可行性。研究對象為東部某大學大學部學生，總樣本數為6,111人。有別於過往學生評鑑教師教學的實證研究皆建構在古典測驗理論，本研究為了更嚴謹地回應學生評鑑教師教學的評鑑結果，因此使用近代測驗理論進行分析，並進一步考量學生的潛在異質差異對學生評鑑教師教學的影響。研究結果顯示，在未考慮評鑑教師的學生潛在異質差異時，教師可輕易通過學校所訂定在學生評鑑教師教學量表的效標門檻。但進一步考慮學生潛在異質性發現，不同潛在類別的學生評鑑教師教學的方式差異頗大。針對上述結果，本研究對大學教師及學生評鑑教師教學提出相關的建議。

This study examines the effects of variability in student ratings regarding instruction on decision-making for faculty teaching evaluation. A total of 6,111 undergraduate students from 173 classes in a university on the east coast of Taiwan were included in the research sample.This study is different from previous studies regarding student ratings for instruction that are constructed in classical test theory. We use item response theory to analyze the heterogeneity of students, to rigorously examine the effects on student ratings regarding instruction. The results show that teachers may easily exceed the teaching criterion score set by the university when not considering the heterogeneity of the student ratings. However, the different latent types of the variability of student ratings may be important for interpreting the results of different student rating scores. The recommendations for university teaching and student ratings regarding instruction are created based on the results from this study.

1. Chang, T.-S.(2005).The validity and reliability of student ratings: Comparison between paper-pencil and online survey.Chinese Journal of Psychology,47(2),113-125.
   連結：
2. Chang, T.-S.,Ross, W.(2006).The influence of instructor, student, and course characteristics on student ratings: What do faculty and students believe?.Journal of National Hualien University of Education,23,1-28.
   連結：
3. 張德勝、邱于真、羅寶鳳(2011)。題目順序對學生評鑑教師教學與學生自評影響之探索性研究。測驗學刊，58(2)，367-389。
   連結：
4. 曾明基、邱于真、張德勝、羅寶鳳(2011)。學生認知歷程對學生評鑑教師教學的影響：階層線性模式分析。課程與教學季刊，14(3)，157-180。
   連結：
5. 曾明基、張德勝、羅寶鳳、邱于真(2012)。學生評鑑教師教學題目安排順序不同對學生評鑑教師的影響： MI 與 MMI 分析取向。測驗學刊，59(1)，131-156。
   連結：
6. Muthén, L. K. (2010, February 11). Re: Model fit index WRMR [Online forum comment]. Retrieved from http://www.statmodel.com/discussion/messages/9/5096.html
7. Beran, T.,Violato, C.(2005).Ratings of university teacher instruction: How much do student and course characteristics really matter?.Assessment & Evaluation in Higher Education,30(6),593-601.
8. Bowling, N. A.(2008).Does the relationship between student ratings of course easiness and course quality vary across schools? The role of school academic rankings.Assessment & Evaluation in Higher Education,33(4),455-464.
9. Brennan, R. L.(Ed.)(2006).Educational measurement.Westport, DC:National Council on Measurement in Education.
10. Celeux, G.,Soromenho, G.(1996).An entropy criterion for assessing the number of clusters in a mixture model.Journal of Classification,13(2),195-212.
11. Cohen, A. S.,Bolt, D. M.(2005).A mixture model analysis of differential item functioning.Journal of Educational Measurement,42(2),133-148.
12. Cohen, E. H.(2005).Student evaluations of course and teacher: Factor analysis and SSA approaches.Assessment & Evaluation in Higher Education,30(2),123-136.
13. d''Apollonia, S.,Abrami, P. C.(1997).Navigating student ratings of instruction.American Psychologist,52(11),1198-1208.
14. Donald, J. G.(2006).Enhancing the quality of teaching in Canada.New Directions for Higher Education,2006(133),23-31.
15. Frühwirth-Schnatter, S.(2006).Finite mixture and markov switching models.New York, NY:Springer.
16. Green, J. L.(Ed.),Camilli, G.(Ed.),Elmore, P. B.(Ed.)(2006).Handbook of complementary methods in education research.Mahwah, NJ:Lawrence Erlbaum Associates.
17. Greenwald, A. G.,Gillmore, G. M.(1997).Grading leniency is a removable contaminant of student ratings.American Psychologist,52(11),1209-1217.
18. Hambleton, R. K.,Swaminathan, H.(1985).Item response theory: Principles and applications.Boston, MA:Kluwer-Nijhoff.
19. Hancock, G. R.(Ed.),Mueller, R. O.(Ed.)(2006).Structural equation modeling: A second course.Greenwich, CT:Information Age.
20. Hancock, G. R.(Ed.),Samuelsen, K. M.(Ed.)(2008).Advances in latent variable mixture models.Charlotte, NC:Information Age.
21. Kreber, C.(2006).Setting the context: The climate of university teaching and learning.New Directions for Higher Education,2006(133),5-11.
22. Masters, G. N.(1985).A comparison of latent trait and latent class analysis of Likert-type data.Psychometrika,50(1),69-82.
23. Masters, G. N.(1988).The analysis of partial credit scoring.Applied Measurement in Education,1(4),279-297.
24. Masters, G. N.(1982).A Rasch model for partial credit scoring.Psychometrika,47(2),149-174.
25. McDonald, R. P.,Mok, M. C.(1995).Goodness of fit in item response models.Multivariate Behavioral Research,30(1),23-40.
26. McKeachie, W. J.(1997).Student ratings: The validity of use.American Psychologist,52(11),1218-1225.
27. McLachlan, G.,Peel, D.(2000).Finite mixture models.New York, NY:John Wiley & Sons.
28. Muraki, E.(1992).A generalized partial credit model: Application of an EM algorithm.Applied Psychological Measurement,16(2),159-176.
29. Muthén, B.(2006).Should substance use disorders be considered as categorical or dimensional?.Addiction,101,6-16.
30. Muthén, B.,Asparouhov, T.(2006).Item response mixture modeling: Application to tobacco dependence criteria.Addictive Behaviors,31(6),1050-1066.
31. Muthén, B.,Asparouhov, T.,Rebollo, I.(2006).Advances in behavioral genetics modeling using Mplus: Applications of factor mixture modeling to twin data.Twin Research and Human Genetics,9(3),313-324.
32. Muthén, L. K.,Muthén, B. O.(2006).Mplus user's guide.Los Angeles, CA:Author.
33. Nylund, K. L.,Asparouhov, T.,Muthén, B. O.(2007).Deciding on the number of classes in latent class analysis and growth mixture modeling: A monte carlo simulation study.Structural Equation Modeling,14(4),535-569.
34. Perry, R. P.(Ed.),Smart, J. C.(Ed.)(2007).The scholarship of teaching and learning in higher education: An evidence-based perspective.Dordrecht, the Netherlands:Springer.
35. Rice, R. E.(2006).Enhancing the quality of teaching and learning: The U.S. experience.New Directions for Higher Education,2006(133),13-22.
36. Rost, J.(1990).Rasch models in latent classes: An integration of two approaches to item analysis.Applied Psychological Measurement,14(3),271-282.
37. Smit, A.,Kelderman, H.,Flier, H.(1999).Collateral information and mixed Rasch models.Methods of Psychological Research Online,4(3),19-32.
38. Spooren, P.,Mortelmans, D.(2006).Teacher professionalism and student evaluation of teaching: Will better teachers receive higher ratings and will better students give higher ratings?.Educational Studies,32(2),201-214.
39. van der Linden, W. J.(Ed.),Hambleton, R. K.(Ed.)(1997).Handbook of modern item response theory.New York, NY:Springer.
40. Vincent, F. F.,Kennon, M. S.(2003).Student psychological need satisfaction and college teacher-course evaluations.Educational Psychology,23(3),235-247.
41. Vincent, F. F.,Kennon, M. S.(2008).Teacher support, student motivation, student need satisfaction, and college teacher course evaluations: Testing a sequential path model.Educational Psychology,28(6),711-724.
42. von Davier, M.,Yamamoto, K.(2004).Partially observed mixtures of IRT models: An extension of the generalized partial-credit model.Applied Psychological Measurement,28(6),389-406.
43. Webler, W. D.(2006).German policy perspectives on enhancing the quality of student learning by university teaching.New Directions for Higher Education,2006(133),53-62.
44. 周祝瑛(2009)。,未出版
45. 張德勝(2002)。學生評鑑教師教學：理論、實務與態度。臺北市=Taipei, Taiwan:揚智=Yang-Chih Book。

1. 曾明基(2019)。縱貫性網路使用行為對學業成就的影響：潛藏轉移模型分析取向。教育科學研究期刊，64(4)，31-59。
2. 曾明基(2020)。學生認知歷程與背景變數對於學生評鑑教師的影響：潛在類別偏差校正與混合迴歸分析。教育科學研究期刊，65(3)，251-276。
3. 曾明基、邱皓政(2015)。研究生評鑑教師教學的結果真的可以與大學生一起比較嗎？多群組混合 MIMIC-DIF 分析。測驗學刊，62(1)，1-23。
4. (2021)。醫學系課程教學評量之調查。高等教育，16(2)，79-111。
5. (2022)。研究生和大學生的學生評鑑教師教學分數真的要一起比較嗎？傾向值結構方程模型分析。教育與心理研究，45(2)，35-57。
6. (2024)。東部某大學的「學生評鑑教師教學量表」分數之排序：貝氏多層次隨機效果模型分析。測驗學刊，71(1)，95-117。