Do Senior High School English Textbooks Sufficiently Prepare Students for the High-Stakes College Entrance Examinations? A Corpus-Based Analysis of Text Difficulty

高中英文教科書是否準備好學生面對高風險的大學入學考試？一個語料庫為本的文本困難度分析研究

10.6481/JTR.202204_15(1).02

程玉秀(Yuh-Show Cheng)；張盛傑(Sheng-Chieh Chang)

corpus-based ； textbook analysis ； vocabulary load ； readability ； 語料庫為本 ； 教科書分析 ； 詞彙量 ； 可讀性

教科書研究

15卷1期（2022 / 04 / 15）

43 - 80

英文

The aim of this study was to uncover how sufficiently senior high school English textbooks in Taiwan prepare students for reading the passages in the high-stakes college entrance examinations in terms of text difficulty. A corpus-based approach was adopted to compare the vocabulary load and readability of passages extracted from senior high school textbooks and from the English test papers of college entrance examinations. Two corpora were compiled: a textbook corpus comprising texts extracted from all five editions of Ministry of Education (MOE)-authorized senior high school textbooks and a test corpus containing all of the reading passages in the college entrance English tests from the 2002 to 2017 school years. The results indicate that the passages in the English textbooks do not match those in the tests in terms of the vocabulary load and several Coh-Metrix readability metrics. The reading passages in the English tests generally have lower overall readability, lower narrativity, and higher syntactic complexity than those in the textbooks. The test passages also require a much larger vocabulary size than the textbooks do. Implications of the findings for students, textbook writers, English teachers, and the MOE are provided.

本研究旨在發掘就文本困難度而言，臺灣高中英文教科書是否準備好學生閱讀大學入學考試英文試題之文本。以語料庫為本的方法，本研究建置了兩個語料庫：教科書語料庫和大學入學考試試題語料庫，進而比較高中教科書和大學入學考試試題英文文本之詞彙量和可讀性。教科書語料庫涵蓋研究期間市面流通的所有審定本高中英文教科書，共五套；試題語料庫則包含了2002年至2017年大學入學考試英文學測和指考的所有篇章。研究結果顯示高中英文教科書和大學入學考試的英文文本，在詞彙量和數個Coh-Metrix可讀性指標上有落差。學測和指考文本的整體可讀性和敘事性通常顯著低於教科書文本，但其語法複雜性則顯著高於教科書文本。閱讀學測和指考試題文本所需的詞彙量也遠高於閱讀教科書文本所需的詞彙量。文末，作者依據研究結果對學生、教科書編輯者、英語教師和教育部提出一些建議。

1. Wang, C.-C. (2008). Communicative language teaching in Taiwan: Teacher conceptions and major challenges. Journal of Applied English, 1, 31-58. https://doi.org/10.29691/JAE.200810.0003
   連結：
2. Baayen, R. H., Piepenbrock, R., & van Rijn, H. (Eds.). (1993). The CELEX lexical database (CD-ROM). Linguistic Data Consortium.
3. Biber, D. (2006). University language: A corpus-based study of spoken and written discourse. John Benjamin.
4. Biber, D. (2010). Corpus-based and corpus-driven analyses of language variation and use. In B. Heine & H. Narrog (Eds.), The Oxford handbook of linguistic analysis (pp. 153-191). Oxford University Press.
5. Carver, R. P. (1994). Percentage of unknown vocabulary words in text as a function of the relative difficulty of the text: Implications for instruction. Journal of Literacy Research, 26(4), 413-437. https://doi.org/10.1080/10862969409547861
6. Chen, H.-L. S., & Huang, H.-Y. (2017). Advancing 21st century competencies in Taiwan. National Taiwan Normal University.
7. Chen, X., & Meurers, D. (2019). Linking text readability and learner proficiency using linguistic complexity feature vector distance. Computer Assisted Language Learning, 32, 418-447. https://doi.org/10.1080/09588221.2018.1527358
8. College Entrance Examination Center. (n.d.-a). GSAT-English. https://www.ceec.edu.tw/en/xmdoc/cont?xsmsid=0J180519944235388511
9. College Entrance Examination Center. (n.d.-b). AST-English. https://www.ceec.edu.tw/en/xmdoc/cont?xsmsid=0J180520414679660023
10. College Entrance Examination Center. (2002). Introduction to reference word list for senior high English education. https://www.ceec.edu.tw/SourceUse/ce37/3.pdf
11. Crossley, S. A., Greenfield, J., & McNamara, D. S. (2008). Assessing text readability using psycholinguistic indices. TESOL Quarterly, 42(3), 475-493.
12. Crossley, S. A., Louwerse, M. M., McCarthy, P. M., & McNamara, D. S. (2007). A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91(1), 15-30. https://doi.org/10.1111/j.1540-4781.2007.00507.x
13. Crossley, S. A., Allen, D. B., & McNamara, D. S. (2011). Text readability and intuitive simplification: A comparison of readability formulas. Reading in a Foreign Language, 23(1), 84-101.
14. Ellis, R. (2005). Principles of instructed language learning. System, 33(2), 209-224. https://doi.org/10.1016/j.system.2004.12.006
15. Frantzen, D. (2010). Incremental gains in foreign language programs: The role of reading in learning about other cultures. Reading in a Foreign Language, 22(suppl. 1), 31-37.
16. Graesser, A. C., Greenberg, D., Olney, A., & Lovett, M. W. (2019). Educational technologies that support reading comprehension for adults who have low literacy skills. In D. Perin (Ed.), The Wiley handbook of adult literacy (pp. 471-493). John Wiley & Sons.
17. Graesser, A. C., & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371-398. https://doi.org/10.1111/j.1756-8765.2010.01081.x
18. Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223-234. https://doi.org/10.3102/0013189X11413260
19. Green, A., Ünaldi, A., & Weir, C. (2010). Empiricism versus connoisseurship: Establishing the appropriacy of texts in tests of academic reading. Language Testing, 27(2), 191-211. https://doi.org/10.1177/0265532209349471
20. Hill, P. (2010). Asia-Pacific secondary education system review series No. 1: Examination systems. United Nations Educational, Scientific and Cultural Organization.
21. Howell, D. C. (2009). Statistical methods for psychology (7th ed.). Cengage Learning.
22. Hsu, W. (2011). The vocabulary thresholds of business textbooks and business research articles for EFL learners. English for Specific Purposes, 30(4), 247-257. https://doi.org/10.1016/j.esp.2011.04.005
23. Klare, G. R. (1974). Assessing readability. Reading Research Quarterly, 10(1), 62-102. https://doi.org/10.2307/747086
24. Koslin, B. I., Zeno, S., & Koslin, S. (1987). The DRP: An effective measure in reading. College Entrance Examination Board.
25. Laufer, B. (1989). What percentage of text lexis is essential for comprehension? In C. Lauren & M. Nordman (Eds.), Special language: From human thinking to thinking machines (pp. 316-323). Multilingual Matters.
26. Laufer, B., & Ravenhorst-Kalovski, G. C. (2010). Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22(1), 15-30.
27. Lexile Framework for Reading. (2016). Lexile analyzer. https://www.lexile.com/tools/lexile-analyzer/step-3-type-or-scan-your-text/
28. Lin, M.-C. (2018). From skill to competence of English language teaching: The contextualized communicative approach. Journal of Education Research, 294, 72-90. https://doi.org/10.3966/168063602018100294005
29. McEnery, T., & Hardie, A. (2011). Corpus linguistics: Method, theory and practice. Cambridge University Press.
30. McNamara, D. S., Graesser, A. C., & Louwerse, M. M. (2012). Sources of text difficulty: Across genres and grades. In J. P. Sabatini, E. Albro, & T. O’Reilly (Eds.), Measuring up: Advances in how we assess reading ability (pp. 89-116). Rowman & Littlefield Education.
31. McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.
32. Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge University Press.
33. Nation, I. S. P. (2006). How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review, 63(1), 59-82. https://doi.org/10.3138/cmlr.63.1.59
34. Nation, I. S. P. (2012). The BNC/COCA word family lists. http://www.victoria.ac.nz/lals/about/staff/publications/BNC_COCA_25000.zip
35. Reynolds, B. L., Shih, Y. C., & Wu, W. H. (2018). Modeling Taiwanese adolescent learners’ English vocabulary acquisition and retention: The washback effect of the College Entrance Examination Center’s reference word list. English for Specific Purposes, 52, 47-59. https://doi.org/10.1016/j.esp.2018.08.001
36. Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26-43. https://doi.org/10.1111/j.1540-4781.2011.01146.x
37. Stenner, A. J. (1996). Measuring reading comprehension with the Lexile framework. https://files.eric.ed.gov/fulltext/ED435977.pdf
38. Sung, Y.-T., Lin, W.-C., Dyson, S. B., Chang, K.-E., & Chen, Y.-C. (2015). Leveling L2 texts through readability: Combining multilevel linguistic features with the CEFR. The Modern Language Journal, 99(2), 371-391. https://doi.org/10.1111/modl.12213
39. Tognini-Bonelli, E. (2001). Corpus linguistics at work. John Benjamins.
40. Webb, S., & Nation, I. S. P. (2008). Evaluating the vocabulary load of written text. TESOLANZ Journal, 16, 1-10.
41. Zhang, X. (2017). A critical review of literature on English language teaching textbook evaluation: What systemic functional linguistics can offer. Journal of Language and Cultural Education, 5(1), 78-102. https://doi.org/10.1515/jolace-2017-0005
42. Zyzik, E. (2009). The role of input revisited: Nativist versus usage-based models. L2 Journal, 1(1), 42-61. https://doi.org/10.5070/l2.v1i1.9056

1. I-Chung Ke(柯宜中)(2023)。Evolution of English Curriculum: A Longitudinal Vocabulary Study of English Textbooks。教科書研究。16(2)。91-130。