Developing Computer-Assisted Speaking Tests for Science and Engineering Students

10.6294/EaGLE.2017.0202.03

Shin-Mei Kao；Chun-Wei Kuo

English for specific academic purposes ； speaking test ； computer-assisted assessment ； engineering and science ； English-medium instruction

English as a Global Language Education (EaGLE) Journal

2卷2期（2017 / 02 / 01）

63 - 90

英文

Due to globalization, many graduate programs in Taiwan offer English medium instruction (EMI) lo encourage enrollment of international students and lo prepare local students for competitive job markets. These programs require evaluation of English oral proficiency for student admission purposes. Current standardized speaking tests are designed to assess general proficiency and are scored manually, which is costly and time-consuming. The present paper describes the development of a computer-assisted speaking test scored using Automatic Speech Recognition technology. The new speaking test was used along with existing standardized speaking, reading, and listening tests with a group of science and engineering graduate students before and after a six-week summer intensive course for specific academic purposes (ESAP). The results show that the new speaking test was highly correlated to the full-length manually scored general speaking test and to the ESAP reading and listening tests taken before the course. The new speaking lest was also correlated to the ESAP reading and listening test after the course. The students also made significant progress in speaking and reading in their professional areas. The student perception questionnaires indicated that the speaking test was well received by the test takers. This study contributed to an improved understanding of the design and implementation of computer-assisted tests used to assess content-related speaking skills.

1. Huang, Y. H.,Tsou, W. L.(2013).Textbook vocabulary knowledge amongst engineering majors in Taiwan.Curriculum & Instruction Quarterly,16(2),201-232.
   連結：
2. Tsou, W. L.,Kao, S. M.(2016).Developing ESAP screening tests for science and engineering.Taiwan International ESP Journal,8(2),1-29.
   連結：
3. British Council. (2017). IELTS: Understand the Speaking Test. Retrieve from http://takeielts.britishcouncil.org/prepare-test/understand-testformat/speaking-test
4. Readable.io. (2012). Retrieved from http://www.readability-score.com/
5. (2013).The Encyclopedia of applied linguistics.Oxford:Wiley-Blackwell..
6. Owen, N. (2012). Can PTE Academic be used as an exit test for a course of academic English? Retrieved from http://pearsonpte.com/wp-content/uploads/2014/07/Owen_Executive_Summary.pdf
7. University of Cambridge Local Examinations Syndicate. (2017). BULATS: Test Format. Retrieved fromhttp://www.bulats.org/paper-based-tests/the-speaking-test
8. L Labs Inc. (2017). MyET. Retrieved from https://www.myet.com/MyETWeb/SubPage.aspx?fn=AboutLLabs.htm
9. Ashwell, T.(2014).Automated scoring for elicited imitation tests.Journal of Global Media Studies,13,37-41.
10. Bachman, L. F.,Palmer, A. S.(1996).Language testing in practice: Designing and developing useful language tests.Oxford:Oxford University Press..
11. Bernstein, J.,De Jong, J. H. A. L.,Pisoni, D.,Townshend, B.(2000).Two experiments on automatic scoring of spoken language proficiency.Proc. of STIL (Integrating Speech Technology in Learning),Dundee, Scotland:
12. Brown, H. G.(2014).Contextual factors driving the growth of undergraduate English-medium instruction programmes at universities in Japan.The Asian Journal of Applied Linguistics,1(1),50-63.
13. Christensen, C.,Hendrickson, R.,Lonsdale, D.(2010).Principled construction of elicited imitation tests.Proceedings of the Seventh Conference on International Language Resources and Evaluation: LREC'10,Valletta, Malta:
14. Costa, F.,Coleman, J. A.(2013).A survey of English-medium instruction in Italian higher education.International Journal of Bilingual Education and Bilingualism,16(1),3-19.
15. Cox, T. L.,Davies, R. S.(2012).Using automatic speech recognition technology with elicited oral response testing.CALICO Journal,29(4),601-618.
16. Coxhead, A.(2000).A new academic word list.TESOL Quarterly,34(2),213-238.
17. Crawford, L.,Tindal, G.(2004).Effects of a read-aloud modification on a standardized reading test.Exceptionality,12(2),89-106.
18. Davies, Alan(Ed.),Elder, Catherine(Ed.)(2004).The handbook of applied linguistics.Hoboken, NJ:Blackwell..
19. Doiz, A.,Lasagabaster, D.,SIERRA, J.(2011).Internationalisation, multilingualism and English‐medium instruction.World Englishes,30(3),345-359.
20. Dolan, R. P.,Hall, T. E.,Banerjee, M.,Chun, E.,Strangman, N.(2005).Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities.Journal of Technology, Learning, and Assessment,3(7)
21. Douglas, D.(2000).Assessing languages for specific purposes.Cambridge:Cambridge University Press..
22. Douglas, D.,Selinker, L.(1992).Analyzing oral proficiency test performance in general and specific purpose contexts.System,20(3),317-328.
23. Erlam, R.(2006).Elicited imitation as a measure of L2 implicit knowledge: An empirical validation study.Applied Linguistics,27(3),464-491.
24. Gabrielatos, C.(2002).,未出版
25. Gallimore, R.,Tharp, R. G.(1981).The interpretation of elicited sentence imitation in a standardized context.Language Learning,31(2),369-392.
26. Graham, C. R.(2006).An analysis of elicited imitation as a technique for measuring oral language proficiency.The Fifteenth International Symposium on English Teaching,Taipei, Taiwan:
27. Graham, C. R.,McGhee, J.,Millard, B.(2010).The role of lexical choice in elicited imitation item difficulty.The 2008 Second Language Research Forum,Somerville, MA:
28. Griffiths, E. J.(2013).UK,University of Exeter.
29. Hamp-Lyons, L.,Lumley, T.(2001).Assessing language for specific purposes.Language Testing,18,127-132.
30. Hsu, W.(2014).Measuring the vocabulary load of engineering textbooks for EFL undergraduates.English for Specific Purposes,33,54-65.
31. Ibrahim, J.(2001).The implementation of EMI (English medium instruction) in Indonesian universities: Its opportunities, its threats, its problems, and its possible solutions.International TEFLIN Conference,Jawa Timur, Indonesia.:
32. Jafari, M.(2013).North Cyprus, Turkey.,Eastern Mediterranean University (EMU)-Doğu Akdeniz Üniversitesi (DAÜ).
33. Jessop, L.,Suzuki, W.,Tomita, Y.(2007).Elicited imitation in second language acquisition research.Canadian Modern Language Review,64(1),215-238.
34. Jin, N. Y.,Ling, L. Y.,Tong, C. S.,Sahiddan, N.,Philip, A.,Azmi, N. H. N.,Tarmizi, M. A. A.(2013).Development of the engineering technology word list for vocational schools in Malaysia.International Education Research,1(1),43-59.
35. Kırkgöz, Y.(2009).Students' and lecturers' perceptions of the effectiveness of foreign language instruction in an English-medium university in Turkey.Teaching in Higher Education,14(1),81-93.
36. Konstantakis, N.(2007).Creating a business word for teaching Business English.Elia: Estudios de Lingüística Inglesa Aplicada,7,79-102.
37. Kramsch, C.(1993).Context and culture in language teaching.Oxford, UK:Oxford University Press..
38. Li, Y.,Qian, D. D.(2010).Profiling the Academic Word List (AWL) in a financial corpus.System,38(3),402-411.
39. Mantali, S. M.,Talib, R.,Mamu, R.(2013).The application of reading aloud technique to increase students' pronunciation.Kim Fakultas Sastra dan Budaya,1(1),1-16.
40. Nakata, T.(2008).English vocabulary learning with word lists, word cards and computers: Implications from cognitive psychology research for optimal spaced learning.ReCALL,20(1),3-20.
41. Nation, I. S. P.(2001).Learning vocabulary in another language.Cambridge, UK:Cambridge University Press.
42. Neri, A.,Cucchiarini, C.,Strik, W.(2003).Automatic speech recognition for second language learning: how and why it actually works.Proc. ICPhS,SA, Australia:
43. Neumeyer, L.,Franco, H.,Digalakis, V.,Weintraub, M.(2000).Automatic scoring of pronunciation quality.Speech Communication,30(2),83-93.
44. Norris, J. M.,Ortega, L.(2000).Effectiveness of L2 instruction: A research synthesis and quantitative meta‐analysis.Language Learning,50(3),417-528.
45. Tarone, E. E.(Ed.),Gass, S. M.(Ed.),Cohen, A. D.(Ed.)(1994).Research methodology in second-language acquisition.Hillsdale, NJ:Lawrence Erlbaum Associates..
46. Tsou, W. L.,Kao, S. M.(2016).Overview of EMI development.2016 Conference on EMI Practices in Higher Education,Tainan, Taiwan.:
47. Ulanoff, S. H.,Pucci, S. L.(1999).Learning words from books: The effects of read-aloud on second language vocabulary acquisition.Bilingual Research Journal,23(4),409-422.
48. Valipouri, L.,Nassaji, H.(2013).A corpus-based study of academic vocabulary in chemistry research articles.Journal of English for Academic Purposes,12(4),248-263.
49. Vinther, T.(2002).Elicited imitation: A brief overview.International Journal of Applied Linguistics,12(1),54-73.
50. Wang, J.,Liang, S. L.,Ge, G. C.(2008).Establishment of a medical academic word list.English for Specific Purposes,27(4),442-458.
51. Weitze, M.,McGhee, J.,Graham, C. R.,Dewey, D. P.,Eggett, D. L.(2011).Variability in L2 acquisition across L1 backgrounds.Selected Proceedings of the 2009 Second Language Research Forum: Diverse Contributions to SLA,Somerville, MA:
52. Wu, W. M.,Stansfield, C. W.(2001).Towards authenticity of task in test development.Language Testing,18(2),187-206.
53. Yan, X.,Maeda, Y.,Lv, J.,Ginther, A.(2016).Elicited imitation as a measure of second language proficiency: A narrative review and meta-analysis.Language Testing,33(4),497-528.
54. Zechner, K.,Higgins, D.,Xi, X.,Williamson, D. M.(2009).Automatic scoring of non-native spontaneous speech in tests of spoken English.Speech Communication,51(10),883-895.