表現標準設定之擴大參與：教學現場效度證據

Extending Participation in Standard Setting: A Validation Study from School Teachers’ Perspective

10.6925/SCJ.201212_8(4).0001

林世華(Sieh-Hwa Lin)；謝佩蓉(Pei-Jung Hsieh)；謝進昌(Jin-Chang Hsieh)

對照組法 ； 表現標準 ； 切截分數 ； 效度覆核 ； contrasting group method ； performance standard ； cut score ； validation

教育研究與發展期刊

8卷4期（2012 / 12 / 31）

1 - 18

繁體中文

近年來，效度證據的思維已更加廣泛而多元。本研究希冀透過「以人為中心」的對照組法，擴大納入教學現場教師的參與，作為2010年採用「以試題為中心」書籤標定法所進行四年級自然科標準設定結果的效度覆核。研究對象為全臺八位國小自然科教師及其任教班級的四年級學生233人。研究工具為自然科表現水準評定表和自然科單一題本標準化測驗，前者供教師逐一評定每位學生的自然科學習表現，屬於基礎以下、基礎、精熟、或者進階之其中一群；後者則是施測於教師所任教班級的學生以連結既有量尺。研究資料採用一般化部分給分模式將學生的二分類作答反應和教師判斷的多分類結果同時估計。結果發現，教師心中所認知最低通過標準，比書籤標定法所得的標準寬鬆，而最高通過標準，則較書籤標定法嚴格。此外，教師判斷整體命中率達52.36%，各表現標準的命中率分別達26.32%、57.14%、58.00%以及54.55%，提供一定程度的外部效度證據。最後提供數項建議供未來研究參考。

The concept of validity evidence has become diverse and multifaceted in recent years. The purpose of the present study is to examine the external validity of science assessment standard setting for 4th grade, which was implemented with the bookmark method in 2010. This study uses ＂contrasting group method＂, an examinee-centered method, to set performance standards. The participants were eight elementary school teachers and their 233 students. The instruments were classification sheet and a particular form of science test. Teachers were instructed to judge the performance of students based on the performance level descriptors and mark in the classification sheet with four levels (basic, basic, proficient, and advanced). In order to link the existing scale and teachers' grading, the particular form of science test was administered to students. Generalized partial credit model was applied to estimate the dichotomous and polytomous data. The results revealed that the minimum standards of basic level set by contrasting group method was lower than that of the bookmark method, while the standard of advanced level by contrasting group method was higher than that of the bookmark method. Besides, the general hit rate was 52.36%, while hit rates of the performance classifications were 26.32%, 57.14%, 58.00%, and 54.55%. In the conclusion, suggestions for further studies are provided.

1. 吳宜芳,鄒慧英,林娟如(2010)。標準設定效度驗證之探究：以大型數學學習成就評量為例。測驗學刊，57(1)，1-27。
   連結：
2. 吳毓瑩,陳彥名,張郁雯,陳淑惠,何東憲,林俊吉(2009)。以常態混組模型討論書籤標準設定法對英語聽讀基本能力標準設定有效性之幅合證據。教育心理學報，41(1)，69-90。
   連結：
3. 杜佳真,林世華(2007)。九年一貫課程數學領域能力指標「數與量」、「代數」主題軸第一、二階段表現標準適切性評估之研究。師大學報：教育類，52(1)，63-85。
   連結：
4. 謝進昌,謝名娟,林世華,林陳涌,陳清溪,謝佩蓉(2011)。大型資料庫國小四年級自然科學習成就評量標準設定結果之效度評估。教育科學研究期刊，56(1)，1-32。
   連結：
5. Bontempo, B. D.,Marks, C. M.,Karabatsos, G.(1998).A meta-analytic assessment of empirical differences in standard setting procedures.annual meeting of the American Educational Research Association,San Diego, CA.:
6. Brandon, P. R.(2002).Two versions of the contrasting-groups standard-setting method: A review.Measurement and Evaluation in Counseling and Development,35(3),167-181.
7. Cizek, G. J.,Bunch, M. B.(2007).Standard setting: A guide to establishing and evaluating performance standards on tests.Thousand Oaks, CA:Sage.
8. Cunningham, G. K.(2005).Must high stakes mean low quality? Some testing program implementation issues.Defending standardized testing,Mahwah, NJ:
9. Educational Testing Service(2002).ETS standards for quality and fairness.Princeton, NJ:Author.
10. Haertel, E. H.(2002).Standard setting as a participatory process: Implications for validation of standards-based accountability programs.Educational Measurement: Issues and Practice,21(1),16-22.
11. Hansche, L. N.(1998).Handbook for the development of performance standards: Meeting the requirements of title I.Bethesda, MD:Frost Associate.
12. Kane, M.(1998).Choosing between examinee-centered and test-centered standard-setting methods.Educational Assessment,5(3),129-145.
13. Kane, M.(1994).Validating the performance standards associated with passing scores.Review of Educational Research,64(3),425-461.
14. Livingston S. A.,Zieky, M. J.(1982).Passing scores: A manual for setting standards of performance on educational and occupational tests.Princeton, NJ:Educational Testing Service.
15. Loomis, S. C.,Bourque, M. L.(2001).From tradition to innovation: Standard setting on the National Assessment of Educational Progress.Setting performance standards,Mahwah, NJ:
16. Muraki, E.(1992).A generalized partial credit model: Application of an EM algorithm.Applied Psychological Measurement,16(2),159-176.
17. Muraki, E.(1993).Information functions of the generalized partial credit model.Applied psychological measurement,17(4),351-363.
18. Näsström, G.,Nyström, P.(2008).A comparison of two different methods for setting performance standards for a test with constructed-response items.Practical Assessment, Research & Evaluation,13(9),1-12.
19. Nichols, P.,Twing, J.,Mueller, C. D.,O'Malley, K.(2010).Standard-setting methods as measurement processes.Educational Measurement: Issues and Practice,29(1),14-24.
20. Nijlen, D. V.,Janssen, R.(2008).Modeling judgments in the Angoff and contrasting-groups method of standard setting.Journal of Educational Measurement,45(1),45-63.
21. Sommers, S.(2012).The training and preparation of Angoff standard setting panelists: The role of group discussion and experience in determining panelist accuracy.1st International Conference on Standard-based Assessment,Taipei:
22. Tannenbaum, R. J.(2011).Setting standards on the Praxis Series Tests: A multistate approach.R&D Connections,17,1-9.
23. Tindal, G.(Ed.),Haladyna, T. M.(Ed.)(2002).Large-scale assessment programs for all students: Validity, technical adequacy, and implementation.Mahwah, NJ:Lawrence Erlbaum Associates.
24. Wolfe, E. W.,Smith, E. V.(2007).Instrument development tools and activities for measure validation using Rasch models: Part I - instrument development tools.Journal of Applied Measurement,8(1),97-123.
25. Wolfe, E. W.,Smith, E. V.(2007).Instrument development tools and activities for measure validation using Rasch models: Part II - validation activities.Journal of Applied Measurement,8(2),234-294.
26. 吳宜芳,鄒慧英(2010)。試題呈現與回饋模式對Angoff標準設定結果一致性提升效益之比較研究。教育研究與發展期刊，6(4)，47-80。
27. 吳裕益(1988)。九種通過分數設定方法之比較研究。初等教育學報，1，47-120。
28. 黃俊傑(2009)。「攜手計畫課後扶助」執行評析及建議。北縣教育，67，69-72。

1. 曾芬蘭、張銘秋、邱佳民(2018)。Yes/No Angoff標準設定結果之效度檢核：應用群聚分析分類法。測驗學刊，65(2)，151-180。