基於ACTEL之華語文寫作評分歸準

Examining the ACTFL Writing Assessment Rating Scale for L2 Chinese Learners

熊玉雯(Yu-Wen Hsiung)；李慧萱(Hui-Hsuan Lee)；宋曜廷(Yao-Ting Sung)

華語文寫作測驗 ； 語言能力等級 ； 評分規準 ； 信度 ； testing L2 Chinese writing ； language proficiency ； ACTFL ； analytic rating scales ； reliability

華語文教學研究

11卷4期（2014 / 12 / 01）

111 - 139

繁體中文

華語文寫作測驗近年來成爲主要大型華語文能力測驗項目之一，其重要性日鉅。本研究以美國外語教學委員會（American Council on theTeaching of Foreign Languages，簡稱ACTFL）所制定之語言分級能力標準為框架，制定出華語文寫作評分規準，並進一步以實證研究來探討ACTFL評分規準所遇到的問題。研究語料為國立臺灣師範大學國語教學中心華語文學習者所寫之同題目作文402篇，專業評分者受培訓後根據ACTFL評分規準評分，研究一方面能為瞭解ACTFL華語文寫作測驗的信度之實徵研究，將評分結果透過百分比一致性與類推性理論分析，檢測測驗之信度。另一方面分析ACTFL之評分架構，就評分規準在分數等級上、計分方式上、分向構面上之三主要研究議題來進行深究與探討，將對寫作測驗有實徵之回饋。

The demand for reliable and valid writing assessing methods of Chinese as a foreign language has grown significantly in recent years. This article reports the construction and the subsequent evaluation of the analytic rating scale based on ACTFL (American Council on the Teaching of Foreign Languages) writing proficiency guidelines. A corpus of 402 writing scripts from different levels at the Mandarin Training Center at National Taiwan Normal University, but all in response to the same writing task, was collected and graded by five professional raters. Each writing test is independently rated by two raters. The validity of the scale is rigidly controlled throughout the construction and raters training process. The grading results provide qualitative analysis of two main points. On the one hand, the reliability of the assessment is examined in terms of the percentage of agreement between raters as well as generalizability and decision studies. On the other, a number of major issues involved in the development of a rating scale are explored for the purpose of this paper, especially on the examination of analytic rating scale against holistic rating scale, as well as the exploration of rating criteria.

1. 王德蕙、李奕璇、曾芬蘭、宋曜廷(2013)。國民中學學生基本學力測驗寫作測驗─信度與效度分析研究。測驗學刊，6(1)，151-184。
   連結：
2. 國家華語測驗推動工作委員會網站。上網日期2013 年3 月5 日。網址 http://www.sc-top.org.tw/chinese/WT/test1.php [Steering Committee for the Test of Proficiency – Huayu. (n. d.). Retrieved March 5, 2013, from :http://www.sc-top.org.tw/chinese/WT/test1.php]
3. 國民中學學生基本學力測驗推動工作委員會。上網日期2013 年3 月5 日。網址http://www.bctest.ntnu.edu.tw/ [The Committee of the Basic Competence Test for Junior High School Students. (n. d.). Retrieved March, 5, 2013, from http://www.bctest.ntnu.edu.tw/]
4. (1987).ACTFL Chinese Proficiency Guidelines.Foreign Language Annals,20(5),471-481.
5. American University Center of Provence. (n. a.). Common European framework of reference: Level breakdown and equivalencies – French. Retrieved May 13, 2013 from http://www.aucp-isha.fr/documents/CEFR_TEF_Levels.pdf
6. ACTFL Proficiency Guidelines 2012 – Writing. 2012. Retrieved August 18, 2014, from http://actflproficiencyguidelines2012.org/writing
7. AP Chinese Language and Culture. Retrieved March 5, 2013, from https://apstudent.collegeboard.org/apcourse/ap-chinese-language-and-culture
8. 中國漢語水準考試。上網日期：2013 年3 月5 日。網址http://www.hsk.org.cn/[HSK: The Chinese Proficiency Test. Retrieved March 5, 2013, from http://www.hsk.org.cn/]
9. Alderson, J. C.(1991).Bands and scores.Language Testing in the 1990s: The Communicative Legacy,London:
10. Bachman, L. F.,Palmer, A. S.(1996).Language Testing in Practice.Oxford:Oxford University Press.
11. Beyreli, L.,Ari, G.(2009).The use of analytic rubric in the assessment of writing performance: Inter-rater concordance study.Educational Sciences: Theory & Practice,9(1),105-125.
12. Brown, G. T. L.,Glasswell, K.,Harland, D.(2004).Accuracy in the scoring of writing: Studies of reliability and validity using a New Zealand writing assessment system.Assessing Writing,9,105-121.
13. Council of Europe(2001).Common European Framework of Reference for Languages: Learning, Teaching, Assessment.Cambridge:Cambridge University Press.
14. DeVellis, R. F.(2002).Scale Development: Theory and Applications.Newbury Park, CA:Sage.
15. East, M.(2009).Evaluating the reliability of a detailed analytic scoring rubric for foreign language writing.Assessing Writing,14,88-115.
16. Fenton, R.,Straugh, T.,Stofflet, F.,Garrison, S.(2000).Improving the validity and reliability of large scale writing assessment.annual meeting of the American Educational Research Association,New Orleans, LA.:
17. Hawkey, R.,Barker, F.(2004).Developing a common scale for the assessment of writing.Assessing Writing,9,122-159.
18. Hughes, A.(2003).Testing for Language Teachers.Cambridge:Cambridge University Press.
19. Knoch, U.(2011).Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from.Assessing Writing,16,81-96.
20. Lumley, T.(2002).Assessment criteria in a large-scale writing test: what do they really mean to the raters?.Language Testing,19(3),246-276.
21. Luoma, S.(2004).Assessing Speaking.Cambridge:Cambridge University Press.
22. Marsh, H. W.,Ireland, R.(1987).The assessment of writing effectiveness: A multidimensional perspective.Australian Journal of Psychology,39,353-367.
23. McNamara, T. F.(1996).Measuring Second Language Performance.London:Longman.
24. Shaw, S. D.,Weir, C. J.(2007).Examining Writing: Research and Practice in Assessing Second Language Writing.Cambridge:Cambridge University Press.
25. Stemeler, S. E.(2004).A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.Practical Assessment Research & Evaluation,9(4)
26. Weigle, S. C.(2002).Assessing Writing.Cambridge:Cambridge University Press.
27. Wigglesworth, G.(ed.),Elder, C.(ed.)(1996).The Language Testing Cycle: From Inceptions to Washback.Melbourne:Australian Review of Applied Linguistics.
28. 方緒軍、楊惠中、朱正才(2011)。語言能力「能做」描述的原理與方案：以CEFR為例。世界語言教學，25(2)，246-257。
29. 林佩樺、彭淑惠、藍珮君(2011)。華語文寫作能力測驗評分者一致性探討。2011 年華語文能力測驗成果發表會，:
30. 柯華葳編(2004)。華語文能力測驗編製：研究與實務。臺北=Taipei:遠流出版公司=Yuan-Liou Publishing Co.。
31. 張郁雯(2009)。華語評量。臺北=Taipei:正中書局=Cheng Chung Book。
32. 張凱編(2006)。漢語水平考試（HSK）研究。北京=Beijing:商務印書館=Commercial Press。
33. 張凱(2002)。語言測驗理論與實踐。北京=Beijing:北京語言文化大學出版社=Beijing Language and Culture University Press。
34. 彭淑惠(2009)。Kaohsiung，國立高雄師範大學華語文教學所=Graduate Institute of Teaching Chinese as a Second/Foreign Language, National Kaohsiung Normal University。
35. 彭淑惠、林美岑、許崴欽。華語文寫作能力測驗信度與評分者一致性探討。華語文能力測驗成果發表會

1. SUNG, Yao-Ting,HONG, Jia-Fei,CHANG, Tao-Hsing(2015).Automatically Detecting Syntactic Errors in Sentences Written by Learners of Chinese as a Foreign Language.中文計算語言學期刊,20(1),49-64.
2. Yu-Chi Liu,Yen-Chiu Chen,Sarah H. J. Liu(2022)。Exploring the Impact of Self-regulated Instruction (SRI) on the CSL Writing of the Automated Essay Scoring System。臺灣華語教學研究，25，41-67。
3. 陳燕秋(2021)。疫情期間AES-Han電腦自動化批改系統對外籍生華語寫作課堂型態所產生的改變與革新。華語文教學研究，18(2)，37-68。
4. 藍珮君,彭淑惠,陳柏熹(2019)。華語文寫作測驗信度與評分規準適切性研究－以流利精通級摘要寫作題型為例。華語文教學研究，16(2)，75-102。
5. 羅曉勤(2019)。『十二年國民基本教育課程綱要:語文領域-第二外國語文』における日本語Level-1の教材開発－台湾若年層学習者の社会．生活文脈を中心に－。台灣日語教育學報，32，105-134。