A Valid and Reliable English Proficiency Exam: A Model from a University Language Program in Taiwan

10.6294/EaGLE.2015.0102.04

James M. Sims

language proficiency test ； language program ； language test construction ； placement exam

English as a Global Language Education (EaGLE) Journal

1卷2期（2015 / 12 / 01）

91 - 125

英文

Assessing language proficiency is not only an important task but also a sensitive issue. This is especially true in Taiwan and much of Asia where testing is strongly ingrained in the culture. There are several commercially produced proficiency exams available on the market today. But, these exams are costly, only offered at limited times, and may not be appropriate for the needs of some language programs. As a result, many universities are in the process of creating their own language proficiency exams. However, there are few models for educational institutions to follow when creating their own proficiency exams. This paper presents the steps a university in Taiwan followed to create an English proficiency exam with a high reliability, appropriate validity and strong correlation to the Test of English as a Foreign Language (TOEFL). This paper includes the six procedures used for developing the language exam: (1) determining the purpose of the test, (2) designing test specifications, (3) constructing test items, (4) evaluating and revising test items, (5) specifying scoring procedures, and (6) performing validity (content, construct, and concurrent) and reliability (split-half and Cronbach's alpha) studies. Finally, the paper concludes with a discussion of the changes to test specifications to better reflex changes in the English ability of current university students in Taiwan. It is hoped that this paper will serve as a model for other schools that want to create their own language proficiency exams.

1. Sims, J.,Liu, J.(2013).Two decades of changes in the English ability of freshmen at a university in Taiwan.Hwa Kang English Journal,19,23-51.
   連結：
2. International Language Testing Association. (2007). Guidelines for practice. Retrieved from http://www.iltaonline.com/images/pdfs/ilta_guidelines.pdf
3. International Test Commission. (2000). International guidelines for test use. Retrieved from https://www.intestcom.org/page/5
4. Alderson, J. C.(2000).Assessing reading.Cambridge, England:Cambridge University Press.
5. Alderson, J. C.,Clapham, C.,Wall, D.(1995).Language test construction and evaluation.Cambridge, England:Cambridge University Press.
6. Bachman, L. F.(2004).Statistical analysis for language assessment.Cambridge, England:Cambridge University.
7. Bachman, L. F.,Palmer, A. S.(1996).Language testing in practice: Designing and Developing Useful Language test.Oxford, England:Oxford University Press.
8. Bae, J.,Bachman, L. F.(1998).A latent variable approach to listening and reading: testing factorial invariance across two groups of children in the Korean/English two-way immersion program.Language Testing,15,380-414.
9. Bailey, K.(1998).Learning about Language Assessment.Boston, MA:Heinle & Heinle.
10. Brindley, G.(1998).Assessing listening abilities.Annual Review of Applied Linguistics,18,171-191.
11. Brown, H. D.(2004).Language assessment: Principles and classroom practices.White Plains, NY:Pearson Education.
12. Buck, G.(2001).Assessing listening.Cambridge, England:Cambridge University Press.
13. Byrne, B. M.(1994).Structural equation modeling with EQS and EQS? Windows.Thousand Oaks, CA:Saga.
14. Chapelle, C. A.,Abraham, R. G.(1990).Cloze method: What does it make?.Language Testing,7,121-146.
15. Cheng, L.(2005).Changing language teaching through language testing: A washback study.Cambridge, England:Cambridge University Press.
16. Cheng, L.(Ed.),Watanabe, Y.(Ed.),Curtis, A.(Ed.)(2004).Washback in language testing: Research contexts and methods.Mahwah, NJ:Erlbaum.
17. Choi, I. C.(2008).The impact of EFL testing on EFL education in Korea.Language Testing,25,39-62.
18. Educational Testing Service(2013).Guidelines for best test development practices to ensure validity and fairness for international english language proficiency assessments.
19. Hale, G. A.,Rock, D. A.,Jirele, T.(1989).Confirmatory factor analysis of the TOEFL. TOEFL research report 32.Princeton, NJ:Educational Testing Service.
20. Henning, G.(1987).A guide to language testing.Los Angeles, CA:Newbury House.
21. Hughes, A.(2002).Testing for language teachers.Cambridge, England:Cambridge University Press.
22. Pan, Y. C.(2013).Does teaching to the exit? A case study of teachers washback in Taiwan.Journal of Asia TEFL,10(4),185-213.
23. Pan, Y. C.(2014).Learner washback variability in standardized exit tests.The Electronic Journal for English as a Second Language,18(2),1-30.
24. Qi, L.(2007).Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high-stakes English test in China.Assessment in Education,14,51-74.
25. Roever, C.,Pan, Y. C.(2008).Test review: GEPT: General English Proficiency Test.Language Testing,25,403-408.
26. Römhild, A.(2008).Investigating the factor structure of the ECPE across different proficiency levels.Spaan Fellow Working Papers in Second of Foreign Language Assessment,6,29-55.
27. Shin, S. K.(2005).Did they take the same test? Examinee language proficiency and the structure of language tests.Language Testing,22,31-57.
28. Shohamy, E.(2001).The power of tests.Harlow, England:Pearson Education.
29. Sims, J.(2012).Perceived changes in the English language ability of freshmen at a university in Taiwan.Journal of Pan-Pacific Association of Applied Linguistics,16(1),33-54.
30. Sims, J.(2008).Tunghai freshmen English for non-majors program.Tunghai Journal of Humanities,49,353-373.
31. Song, M. Y.(2008).Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach.Language Testing,25,435-464.
32. Tomblin, J. B.,Zhang, X.(2006).The dimensionality of language ability in school-age children.Journal of Speech, Language, and Hearing Research,49,1193-1208.
33. Tsai, Y.,Tsou, C. H.(2009).A standardized English Language Proficiency test as the graduation benchmark: Student perspectives on its application in higher education.Assessment in Education: Principles, Policy & Practice,16,319-330.
34. Widdowson, H. G.(Ed.)(2000).Language testing.Oxford, England:Oxford University Press.