英語能力測驗難度的控管－以臺師大英語會考為例

How to Control the Level of Difficulty of an English Proficiency Test

10.6249/SE.2015.66.1.07

陳秋梅(Chiou-Mei Chen)

能力測驗 ； 試題難度／通過率 ； 難易度變數 ； 鑑別度 ； proficiency test ； facility value ； level of difficulty ； variables of difficulty ； variables of facility ； discriminability

中等教育

66卷1期（2015 / 03 / 01）

136 - 152

繁體中文

本研究分為二大目的，目的一：呈現本人近年來對難度變數的研究發現；目的二：與有志於撰寫能力測驗者分享本人多年實務經驗的心得，希望藉此協助他們控管試題的品質。其實要控制試題難度並不是一件容易的事，因為難度並不是一個不變的定數，它會隨著考生的程度、考試的情境以及試題本身的種種因素而改變，命題者需要累積許多經驗才能正確地判斷變數運作的方向和力道。本研究的資料取自臺師大三年的英語會考試題，考生為非英語系的大一新生，程度從高級到初級皆有，共超過2700人，試題共285題，經統計考驗，試題信度都超過0.82。研究步驟首先透過文獻探討，列出常見會影響閱讀測驗和聽力測驗通過率的變數。接著把所有試題按照通過率高低排列，逐一列舉並審視每一題牽涉到的變數的種類和數目，結果發現了以下的趨勢－試題包含的難度變數愈多，難度就愈高。但其中有一些例外，本研究也進一步檢視並分析其原因。最後，根據分析的結果，提出一些命題原則和步驟，同時還設計出一份可以粗略計算難度的表格，供命題者參考。

For many test writers, it is common sense that the level of difficulty must be appropriate in order to discriminate test takers of different English levels. However, to know is one thing, and to do is another. According to my past experience as a test reviewer, quite a few test writers failed to put their testing knowledge into real practice, so the items they wrote were not good enough in terms of either facility or discriminability. In response to this gap, this study is intended to explore how to well control the facility value of a test. In my opinion, the failure of the test writers to control the level of difficulty can be largely attributed to a misconception, i.e. they regard it as something static rather than dynamic. In fact, the reverse is true. The facility value always varies with a lot of factors, such as the test taker’s English competence, the testing environment and condition, and a variety of variables that are involved with the test itself. Therefore test writers need much experience so that they can correctly judge which variables will be active enough to affect the test result, while others will not. The data for this study comprise 3 English proficiency tests administered to all the non-English-major freshmen of NTNU from 2003 to 2005. The 3 tests are all statistically proved to be reliable. There are altogether 285 items and over 2700 subjects with different levels of English abilities. To begin with, this study lists all the relevant variables of facility/difficulty through literature survey. Next, all the items are arranged in order of their facility values, and each of them is scrutinized to see what variables of facility or difficulty are involved, how these variables interact with each other, and if there is any hierarchical relationship among the variables. A general tendency for predicting the facility value is found. Finally, some specific guidelines are proposed based on the result of the analysis. Besides, a table listing all the relevant variables of facility / difficulty is provided for test writers to code or measure the level of difficulty.

1. (1995).Collins Cobuild English Dictionary.London:Collins.
2. Abraham, R. G.,Chapelle, C. A.(1992).The meaning of cloze test scores: an item difficulty perspective.The Modern Language Journal,76(4),468-79.
3. Abrahamsen, E.,Shelton, K.(1989).Reading comprehension in adolescence with learning disabilities: semantic and syntactic effects.Journal of Learning Disabilities,22,569-72.
4. Chen, Chiou-mei(2006).Are inference items more difficult than non-inference items in EFL listening comprehension tests?.Selected Papers from the Fifteenth International Symposium on English Teaching
5. Chen, Chiou-mei(2009).Examination of the variables of difficulty in EFL listening and reading tests.The Proceedings of 2009 International Conference and Workshop on TEFL and Applied Linguistics
6. Chiang, C. S.,Dunkel, P.(1992).The effect of speech modification, prior knowledge, and listening proficiency on EFL lecture learning.TESOL Quarterly,26,345-74.
7. Davey, B.(1988).Factors affecting the difficulty of reading comprehension items for successful and unsuccessful readers.Journal of Experimental Education,56,67-76.
8. Drum, P. A.,Calfee, R. C.,Cook, L. K.(1981).The effects of surface structure variables on performance in reading comprehension tests.Reading Research Quarterly,16,486-514.
9. Freedle, R.,Kostin, I.(1996).TOEFL Research Report 56TOEFL Research Report 56,Princeton, NJ:ETS.
10. Freedle, R.,Kostin, I.(1993).The prediction of TOEFL reading item difficulty: implications for construct validity.Language Testing,10,133-70.
11. Freedle, R.,Kostin, I.(1991).ETS Research Report RR-91-29ETS Research Report RR-91-29,Princeton, NJ:Educational Testing Service.
12. Freedle, R.,Kostin, I.(1992).ETS Research Report RR-91-59ETS Research Report RR-91-59,Princeton, NJ:Educational Testing Service.
13. Freedle, R.,Kostin, I.(1999).Does the text matter in a multiple-choice test of comprehension? The case for the construct validity of TOEFL's minitalks.Language Testing,16(1),2-32.
14. Hare, V.,Rabinowitz, M.,Schieble, K.(1989).Text effects on main idea comprehension.Reading Research Quarterly,24,72-88.
15. Nissan, S.(1996).TOEFL Research Report RR-95-37TOEFL Research Report RR-95-37,Princeton, NJ:ETS.
16. Pollitt, A.,Hutchinson, C.,Entwistle, N.,Deluca, C.(1985).What Makes Exam Questions Difficult: an Analysis of "0" Grade Questions and Answers.Edinburgh:Scottish Academic Press.
17. Shohamy, E.,Inbar, O.(1991).Validation of listening comprehension tests: the effect of text and question type.Language Testing,8(1),23-40.
18. 陳秋梅(2005)。影響閱讀測驗難易度變數之再探。第二十二屆中華民國英語文教學研究國際研討會論文集，臺北:
19. 陳秋梅(2005)。影響聽力測驗難易度變數之再探。第十四屆中華民國英語文教學國際研討會論文集，臺北:
20. 語言訓練測驗中心(2000)。,臺北:語言訓練測驗中心。
21. 語言訓練測驗中心(1999)。,臺北:語言訓練測驗中心。
22. 鄭恆雄(2002)。大學入學考試中心2002年研究報告大學入學考試中心2002年研究報告,臺北:大學入學考試中心。
23. 鄧慧君(1998)。文類及題型對英語聽力測驗之影響。英語教學，23(1)，5-18。