「國民中學學生基本學力測驗寫作測驗」信度與效度分析研究

The Reliability and Validity ofWriting Assessment of Basic Competence Test for Junior High School Students

王德蕙(Te-Hui Wang)；李奕璇(Yi-Hsuan Lee)；曾芬蘭(Fen-Lan Tseng)；宋曜廷(Yao-Ting Sung)

信度 ； 效度 ； 國民中學學生基本學力測驗 ； 寫作測驗 ； basic competence test for junior high school students ； reliability ； validity ； writing assessment

測驗學刊

60卷1期（2013 / 03 / 01）

151 - 185

繁體中文

「國民中學學生基本學力測驗」自2007年正式加考寫作測驗以來，外界對於寫作測驗的評分者一致性，以及每次只考一個題目是否足以評量到考生的寫作能力，始終非常的關心。加上國中基測屬於高風險測驗，考生人數眾多，影響極鉅，以及國內外在寫作測驗信效度研究上的不足，因此實有必要進行寫作測驗信效度之研究。本研究之研究目的有二：首先為探討「國民中學學生基本學力測驗寫作測驗」（簡稱「國中基測寫作測驗」）的信度，以類推性理論為主，輔以評分者一致率與評分者間相關等數據，作為驗證信度的依據；其次，蒐集命題設計、評分流程、寫作分數與其他效標的關聯等各種效度證據，以驗證寫作測驗之效度。期望透過本研究結果，一方面能提供測驗實務界關於高風險考試中如何監控與提升寫作測驗信度與效度的作法，另一方面可以讓社會相關人士了解「國中基測寫作測驗」的信度、效度之實徵證據。而研究結果也顯示「國中基測寫作測驗」確實具有良好之信度與效度。

Since writing assessment of Basic Competence Test for Junior High School Students (BCTEST-WA) has been implemented from 2007, its reliability and validity has been one of the most concerning issues to public. Due to the reason that BCTEST-WA is a large-scale high-stake exam, but existing researches on reliability and validity of writing assessment are insufficient, the purpose of this study is to validate the reliability and validity of BCTEST-WA. Three approaches to validate the reliability include generalizability theory, raters’ consensus, and raters’ consistency. The exam validity was illustrated through collecting evidences based on test content, responses processes, and relations to other variables. The results show that BCTEST-WA, as a large-scale highstake exam, has high reliability and validity. This paper does not only present the evidences on the good reliability and validity of BCTEST-WA, but it also provides references on how to monitor and improve reliability and validity of high-stake writing assessments.

1. Educational Testing Service (n.d.). TOEFL writing rubrics. Retrieved from http://www.ets.org/Media/Tests/TOEFL/pdf/Writing_Rubrics.pdf
2. Illinois State Board of Education (n.d.). Illinois learning standards. Retrieved from http://www.isbe.net/ils/ela/standards.htm
3. United States Department of Education (n.d.). What does the NAEP writing assessment measure?Retrieved from http://nces.ed.gov/nationsreportcard/writing/whatmeasure.asp
4. Navada Department of Education (n.d.). Writing assessment background and purpose. Retrieved from http://nde.doe.nv.gov/Assessment_WA_BGPurpose.htm
5. Tennessee Department of Education (n.d.). Writing assessment. Retrieved from http://www.state.tn.us/education/assessment/writing.shtml
6. American Educational Research Association,American Psychological Association,National Council on Measurement in Education(1999).Standards for educational and psychological testing.Washington, DC:Author.
7. Bereiter, C.,Scardamalia, M.(1987).The psychology of written composition.Hillsdale, NJ:Lawrence Erlbaum Associates.
8. Brennan, R. L.(2001).Generalizability theory.New York, NY:Springer-Verlag.
9. Brennan, R. L.(2000).Performance assessments from the perspective of generalizability theory.Applied Psychological Measurement,24(4),339-353.
10. Brown, G. T. L.,Glasswell, K.,Harland, D.(2004).Accuracy in the scoring of writing: Studies of reliability and validity using a New Zealand writing assessment system.Assessing Writing,9,105-121.
11. Bunch, M. B.,Littlefair,W.(1988).Total score reliability in large-scale writing assessment.conference of the Education Commission of the States,CO.:
12. Chen, E.,Niemi, D.,Wang, J.,Wang, H.,Mirocha, J.(2007).,Los Angeles, CA:National Center for Research on Evaluation, Standards, and Student Testing, University of California.
13. Crocker, L.,Algina, J.(1986).Introduction to classical and modern test theory.Orlando, FL:Holt, Rinehart and Winston.
14. Fenton, R.,Straugh, T.,Stofflet, F.,Garrison, S.(2000).Improving the validity and reliability of large scale writing assessment.annual meeting of the American Educational Research Association,New Orleans, LA.:
15. Gao, X.,Brennan, R. L.(2001).Variability of estimated variance components and related statistics in a performance assessment.Applied Measurement in Education,14(2),191-203.
16. Huot, B.(1990).Reliability, validity, and holistic scoring: What we know and what we need to know.College Composition and Communication,41,201-213.
17. Kretchmar, J.(2006).Assessing the reliability of ratings used in undergraduate admission decisions.Journal of College Admission,192,10-15.
18. Kroll, B.(Ed.)(1990).Second language writing: Research insights for the classroom.Cambridge, UK:Cambrige University.
19. Novak, J. R.,Herman, J. L.,Gearhart, M.(1996).Establishing validity for performancebased assessments: An illustration for collections of student writing.The Journal of Educational Research,89(4),220-233.
20. Shavelson, R. J.,Webb, N. M.(1991).Generalizability theory: A primer.Newbury Park, CA:Sage.
21. Stemler, S. E.(2004).A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.Practical Assessment Research & Evaluation,9(4)
22. Traub, R. E.,Rowley, G. L.(1991).Understanding reliability.Educational Measurement: Issues and Practice,10(1),37-45.
23. Wagner, D. A.(2004).Literacy(ies), Culture(s), and development(s): The ethnographic challenge.Reading Research Quarterly,39,234-241.
24. 朱曉斌(2007)。寫作教學心理學。中國杭州:浙江大學出版社。
25. 祝新華(1995)。語文能力結構研究。教育研究，15，54-60。
26. 劉雨(1995)。寫作心理學。高雄市:麗水文化。
27. 謝錫金(1984)。高中學生的寫作思維過程。語文雜誌，12，41-53。

1. 藍珮君,彭淑惠,陳柏熹(2019)。華語文寫作測驗信度與評分規準適切性研究－以流利精通級摘要寫作題型為例。華語文教學研究，16(2)，75-102。
2. 熊玉雯、李慧萱、宋曜廷(2014)。基於ACTFL之華語文寫作評分規準。華語文教學研究，11(4)，111-139。
3. 徐志輝、柳立偉(2016)。銀髮族投入休閒活動對健康促進生活型態之影響。嘉大體育健康休閒期刊，15(1)，78-91。