题名

華語文寫作測驗信度與評分規準適切性研究-以流利精通級摘要寫作題型為例

并列篇名

The Reliability and Rubric Relevance of the TOCFL Writing Test - Analyzing Band C Summary Writing as an Example

作者

陳柏熹(Po-Hsi CHEN);彭淑惠(Shu-Hui PENG);藍珮君(Pei-Jiun LAN)

关键词

信度 ; 流利精通級 ; 華語文寫作測驗 ; 評分規準 ; 摘要寫作 ; Band C Test ; reliability ; rubrics ; Summary Writing ; TOCFL Writing Test

期刊名称

華語文教學研究

卷期/出版年月

16卷2期(2019 / 06 / 01)

页次

75 - 102

内容语文

繁體中文

中文摘要

寫作測驗因題數少且須仰賴人工評分,使其信度研究更形重要,然而過往華語寫作測驗信度相關文獻甚為少見。緣此,本研究針對流利精通級華語文寫作測驗摘要寫作題型的信度與評分規準的適切性進行探討,運用的分析模式包括斯皮爾曼等級相關分析、類推性理論與多元迴歸分析。研究結果顯示:(1)在採取分析式評分方式的情況下,多數評分者所評定出的整體級分與最後成績呈中度或高度正相關,評分者間信度大致良好。(2)受試者的變異成分最高,可佐證其得分能反映寫作能力。(3)文本由3人評閱,有利兼顧評分品質與經濟效益。(4)在11個變項中,複雜句型、組織、詞彙語法、詞語運用、規範性為預測力較為顯著的變項。

英文摘要

A writing test is a form of language test that has fewer items in each test and relies heavily on subjective rating. These features of the writing test make the issue of reliability rather significant. This research focuses on the task type of Summary Writing used in the TOCFL Writing Test, Band C. By utilizing the analysis methods associated with Spearman's rank correlation coefficient, multiple regression analysis and generalizability theory, this study aims to discuss the reliability of, and the rubrics associated with, this specific task type. The results of this study can be summarized as follows: (1) By adopting an analytic rating method, the individual scores given by the majority of the raters have high-positive or medium-positive correlations with the final score. Inter-rater reliability is moderately high. (2) The variance components of the test-takers are the highest, which indicates that the scores for this task type can clearly reflect individual test-taker's writing ability. (3) Assigning three raters to rate each test essay can assure the quality and efficiency of the rating process. (4) Among the 11 variables which we have analyzed, 5 of them - complex sentence patterns, organization, vocabulary and syntax, vocabulary switching, and response requirements - can predict test-takers' writing ability more significantly.

主题分类 人文學 > 語言學
社會科學 > 教育學
参考文献
  1. 王德蕙, De-hui,李奕璇, Yi-xuan,曾芬蘭, Fen-lan,宋曜廷, Yao-ting(2013)。國民中學學生基本學力測驗寫作測驗-信度與效度分析研究。測驗期刊,6(1),151-184。
    連結:
  2. 熊玉雯, Yu-wen,李慧萱, Hui-hsuan,宋曜廷, Yao-ting(2014)。基於 ACTFL 之華語文寫作評分規準。華語文教學研究,11(4),111-139。
    連結:
  3. Alderson, Charles J.,Clapham, Caroline,Wall, Dianne(1995).Language Test Construction and Evaluation.Cambridge:Cambridge University Press.
  4. Alkharusi, Hussain(2012).Generalizability theory: An analysis of variance approach to measurement problems in educational assessment.Journal of Studies in Education,2(1),184-196.
  5. Bachman, Lyle F.(1990).Fundamental Considerations in Language Testing.Oxford:Oxford University Press.
  6. Brennan, Robert L.(2000).Performance assessments from the perspective of generalizability theory.Applied Psychological Measurement,24(2),339-353.
  7. Brookes, Arthur,Grundy, Peter(1998).Beginning to Write.Cambridge:Cambridge University Press.
  8. Chang, Li-ping(2017).The development of the test of Chinese as a foreign language (TOCFL).Assessing Chinese as a Second Language,Berlin:
  9. Chen, Eva,Niemi, David,Wang, Jia,Wang, Haiwen,Mirocha, Jim(2007).,Los Angeles, CA:National Center for Research on Evaluation, Standards, and Student Testing, University of California.
  10. Council of Europe(2001).Common European Framework of Reference for Languages: Learning, Teaching, Assessment.Cambridge:Cambridge University Press.
  11. Grabe, William,Kaplan, Robert B.(1996).Theory and Practice of Writing.NY:Longman.
  12. Kretchmar, Jennifer(2006).Assessing the reliability of ratings used in undergraduate admission decisions.Journal of College Admission,192,10-15.
  13. Luoma, Sari(2004).Assessing Speaking.Cambridge:Cambridge University Press.
  14. Mushquash, Christopher,O’Connor, Brian P.(2006).SPSS and SAS programs for generalizability theory analysis.Behavior Research Methods,38,542-547.
  15. Shavelson, Richard J.,Webb, Noreen M.(1991).Generalizability Theory: A primer.Newbury Park, AC:Sage.
  16. Shavelson, Richard J.,Webb, Noreen M.,Rowley, Glenn L.(1989).Generalizability theory.American Psychologist,44(6),922-932.
  17. Sullivan, Kathleen E.(1980).Paragraph Practice: Writing the Paragraph and the Short Composition (4th edition).NY:MacMillan Publishing Co..
  18. Webb, Noreen M.,Rowley, Glenn L.,Shavelson, Richard J.(1988).Using generalizability theory in counseling and development.Measurement and Evaluation in Counseling and Development,21,81-90.
  19. Weigle, Sara C.(2002).Assessing Writing.Cambridge:Cambridge University Press.
  20. 王文中, Wen-chung,呂金燮, Chin-hsieh,吳毓瑩, Yuh-yin,張郁雯, Yu-wen,張淑慧, Shu-hui(2008).教育測驗與評量-教室學習觀點.臺北=Taipei:五南出版社=Wu-Nan Book Inc..
  21. 吳明隆, Ming-lung(2000).SPSS 統計運用實務.臺北=Taipei:松崗電腦圖書資料股份有限公司=Unalis Corporation.
  22. 國家華語測驗推動工作委員會=Steering Committee for the Test Of Proficiency-Huayu(2015)。,新北市=New Taipei:國家華語測驗推動工作委員會=SC-TOP。
  23. 國家華語測驗推動工作委員會=Steering Committee for the Test Of Proficiency-Huayu(2016)。,新北市=New Taipei:國家華語測驗推動工作委員會=SC-TOP。
  24. 國家華語測驗推動工作委員會=Steering Committee for the Test Of Proficiency-Huayu(2018)。,新北市=New Taipei:國家華語測驗推動工作委員會=SC-TOP。
  25. 張玉茹, Yu-ju(2004)。臺北=Taipei,國立臺灣師範大學=National Taiwan Normal University。
  26. 張郁雯, Yu-wen(2009).華語評量.臺北=Taipei:正中書局=Cheng Chung Book Co., Ltd..
  27. 陳柏熹, Po-hsi(2011).心理與教育測驗:測驗編製理論與實務.臺北=Taipei:精策教育=Planned Education Ltd..
  28. 聶丹, Dan(2009)。漢語水平考試(HSK)寫作評分標準發展概述。雲南師範大學學報(對外漢語教學與研究版),7(6),15-20。