题名

On What Rational Cloze Tests Can Measure: A Revisit Applying Confirmatory Factor Analysis

并列篇名

理性刪除克漏字測驗效度的重新審視：驗證性因素分析方法的應用

DOI

10.6384/CIQ.201607_19(3).0008

作者

林甫雯(Fu-Wen Lin)；林文鶯(Wen-Ying Lin)；游錦雲(Ching-Yun Yu)

关键词

理性刪除克漏字測驗 ； 驗證性因數分析方法 ； 二階因素 ； 銜接理解能力 ； 貫理解能力 ； rational cloze tests ； confirmatory factor analysis ； second-order factor ； cohesive competence ； coherent competence

期刊名称

課程與教學

卷期/出版年月

19卷3期（2016 / 07 / 01）

页次

205 - 236

内容语文

英文
中文摘要

此研究旨在證實容易編寫的理性刪除克漏字測驗（rational cloze test）可以很有信度及效度地用來測試課堂上學生的各種不同閱讀技能。此研究藉由驗證性因數分析方法（confirmatory factor analysis），重新探討Bachman（1982）所提出的主張：理性刪除克漏字測驗是可以用來測試課堂上學生的各種不同閱讀技能，尤其是銜接理解能力及連貫理解能力。713位來自臺灣北部四所大學的同學，在研修一般性英文課程的課堂上，填答了Bachman（1985）所編寫一份的理性刪除克漏字測驗。其結果顯示本研究的數據與整體特質加四個細項特質的二階模式比較吻合；也就是說，理性刪除克漏字測驗是可以用來測試課堂教師想要考的各種不同閱讀技能，如文法能力、跨子句的銜接及連貫理解能力、跨句子的銜接及連貫理解能力、及篇章文字線索外的推論能力，而且這四個細項能力可以用二階整體特質解釋。
英文摘要

In response to the demand for quality tests that are easy to construct and in view of the cohesive and coherent competence that is important to reading comprehension and writing development, the present study was motivated and conducted, in the hope that its results could prove informative as to whether easily constructed rational cloze tests could be customized for classroom language teachers' testing objectives with high reliability and construct validity. Employing the confirmatory factor analysis (CFA), the current study aimed to investigate whether additional evidence can be obtained to support Bachman's (1982) claim that the rational cloze test can be designed by classroom language teachers to measure an array of reading skills as intended, specifically including cohesive and coherent competence. In this study were a total of 713 participants, all college students, taking the general English courses at four universities in northern Taiwan. The participants' dichotomously-scored responses to a rational cloze test constructed by Bachman (1985) were analyzed through the CFA on the item-by-item level in the program Mplus, which provides a convenient mechanism to perform the CFA of dichotomous responses. The present study, partly following Bachman (1982), tested the general trait model, the four specific traits model, and the general plus specific traits model-the last comprising a general factor plus the four specific traits. The testing results of the present study have lent support to the hypothesized general plus specific traits model, stating that rational cloze tests can be designed to tap distinct and related language competence, such as syntactic competence, the cohesive and coherent competence that depends on contexts of across-clauses and across-sentences, and extra-textual inference, all of which could be explained by the second-order general language proficiency factor. Based on the findings of the present study, implications and recommendations were provided for future research and classroom language teachers, as well as language test constructors.
主题分类 社會科學 > 教育學
参考文献
  1. Hampton, S. (2010). The importance of writing structures, coherence, and cohesion to writing and reading. Retrieved from http://www.reading.org/libraries/book-supplements/bk767supp-hampton.pdf
  2. Min, Y.-K. (n.d.). ESL: Coherence and cohesion. Retrieved from http://www.bothell.washington.edu/wacc/for-students/eslhandbook/coherence
  3. Selivan, L. (2013, May 5). In context or with co-text? [Web log comments]. Retrieved from http://leoxicon.blogspot.tw/2013/05/context-or-cotext.html
  4. León, D. A. D. (2011). Anχálise factorial confirmatória através dos softwares Re Mplus (unpublished manuscript). Universidade Federal do Rio Grande do Sul, Porto Alegre, RS..
  5. Fries, C. C. (1945). Teaching and learning English as a foreign language. Ann Arbor, MI: University of Michigan Press.
  6. Cambridge English Language Assessment. (2015). Cambridge English research and validation. Retrieved from http://www.cambridgeenglish.org/principles/
  7. Alderson, J. C.(1979).The cloze procedure and proficiency in English as a foreign language.TESOL Quarterly,13,219-227.
  8. Anderson, D. C.(1980).University of Illinois.
  9. Bachman, L. F.(1985).Performance on cloze tests with fixed-ratio and rational deletions.TESOL Quarterly,19,535-555.
  10. Bachman, L. F.(1982).The trait structure of cloze test scores.TESOL Quarterly,16,61-70.
  11. Bachman, L. F.,Palmer, A. S.(1982).The construct validation of some components of communicative proficiency.TESOL Quarterly,16,449-465.
  12. Bachman, L. F.,Palmer, A. S.(1981).The construct validation of the FSI oral interview.Language Learning,31,67-86.
  13. Bagozzi, R. P.,Yi, Y.(1988).On the evaluation of structural equation models.Journal of the Academy of Marketing Science,16(1),74-94.
  14. Bandalos, D. L.(2002).The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling.Structural Equation Modeling,9,78-102.
  15. Beinborn, L.,Zesch, T.,Gurevych, I.(2015).Candidate evaluation strategies for improved difficulty prediction of language tests.Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications,Stroudsburg, PA:
  16. Bollen, K. A.(1989).Structural equations with latent variables.New York, NY:John Wiley & Sons.
  17. Bollen, K. A.(1980).Issues in the comparative measurement of political democracy.American Sociological Review,45,370-390.
  18. Boshrabadi, A. M.,Biria, R.,Hodaeian, M.(2014).A contrastive analysis of the links of textuality in abstracts written by Persian and English writers in clinical psychology journals.International Journal of Applied Linguistics and English Literature,3(4),136-142.
  19. Burns, G. L.,Patterson, D. R.(2000).Factor structure of the Eyberg Child Behavior Inventory: A parent rating scale of oppositional defiant behavior toward adults, inattentive behavior, and conduct problem behavior.Journal of Clinical Child Psychology,29(4),569-577.
  20. Cai, H.(2013).Partial dictation as a measure of EFL listening proficiency: Evidence from confirmatory factor analysis.Language Testing,30(2),177-199.
  21. Carroll, J. B.(1975).The teaching of French as a foreign language in eight countries.New York, NY:Halsted.
  22. Chapman, I. J.(1983).Reading development and cohesion.London:Hememan.
  23. Chen, F. F.,Sousa, K. H.,West, S. G.(2005).Testing measurement invariance of second-order factor models.Structural Equation Modeling,12(3),471-492.
  24. Cooper, C. R.(Ed.),Odell, L.(Ed.)(1999).Evaluating writing: The role of teacher's knowledge about text, learning, and culture.Urbana, IL:National Council of Teachers of English.
  25. Crowhurst, M.(1987).Cohesion in argument and narration at three grade levels.Research in the Teaching of English,21,185-201.
  26. Diamantopoulos, A.,Siguaw, J. A.(2000).Introducing LISREL: A guide for the uninitiated.Thousand Oaks, CA:Sage.
  27. Dimitrov, D. M.(2010).Testing for factorial invariance in the context of construct validation.Measurement and Evaluation in Counseling and Development,43(2),121-149.
  28. DiStefano, C.,Hess, B.(2005).Using confirmatory factor analysis for construct validation: An empirical review.Journal of Psychoeducational Assessment,23,225-241.
  29. Farhady, H.,Keramati, M. N.(1996).A text-driven method for the deletion procedure in cloze passages.Language Testing,13,191-207.
  30. Flood, J.(Ed.)(1984).Understanding reading comprehension.Newark, DE:International Reading Association.
  31. Fornell, C.,Larcker, D. F.(1981).Evaluating structural equation models with unobservable variables and measurement error.Journal of Market Research,18,39-50.
  32. Fouly, K. A.,Bachman, L. F.,Cziko, G. A.(1990).The divisibility of language competence: A confirmatory approach.Language Learning,40(1),1-21.
  33. Frank, M.(1993).Modern English: A practical reference guide.Englewood Cliffs, NJ:Regen/Pearson Hall.
  34. Fulcher, G.(1989).Cohesion and coherence in theory and reading research.Journal of Research in Reading,12(2),146-163.
  35. Gardner, R.,Lambert, W. E.(1965).Language aptitude, intelligence, and second language achievement.Journal of Educational Psychology,65,191-227.
  36. Halliday, M. A. K.,Hasan, R.(1976).Cohesion in English.London:Longman.
  37. Harsch, C.,Hartig, J.(2015).Comparing C-tests and Yes/No vocabulary size tests as predictors of receptive language skills.Language Testing,32(1),1-21.
  38. Hoey, M.(1991).Patterns of lexis in text.Oxford:Oxford University Press.
  39. Horning, A.(1991).Readable writing: The role of cohesion and redundancy.Journal of Advanced Composition,11(1),135-145.
  40. Hosley, D.,Meredith, K.(1979).Inter- and intra-test correlates of the TOEFL.TESOL Quarterly,13,209-217.
  41. Hu, L.,Bentler, P. M.(1999).Cutoff criterion for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives.Structural Equation Modeling,6,1-55.
  42. Irwin, J.(Ed.)(1986).Understanding and teaching cohesion comprehension.Newark, DE:International Reading Association.
  43. Jöreskog, K.,Sörbom, D.(1993).LISREL 8: Structural equation modeling with the SIMPLIS command language.Hillsdale: NJ:Lawrence Erlbaum Associates, Inc..
  44. Kano, Y.,Azuma, Y.(2003).Use of SEM programs to precisely measure scale reliability.New developments in psychometrics,Tokyo, Japan:
  45. Khodadady, E.(2012).Validity and tests developed on reduced redundancy, language components and schema theory.Theory and Practice in Language Studies,2(3),585-595.
  46. Kim, S.,Hagtvet, K. A.(2003).The impact of misspecified item parceling on representing latent variables in covariance structure modeling: A simulation study.Structural Equation Modeling,10,101-127.
  47. Kintsch, W.(1988).The use of knowledge in discourse processing: A constructionintegration model.Psychological Review,95,163-182.
  48. Klein-Braley, C.(1997).C-Tests in the context of reduced redundancy testing: An appraisal.Language Testing,14,47-84.
  49. Klein-Braley, C.(1981).Duisburg, Germany,University of Duisburg.
  50. Lado, R.(1964).Language teaching: A scientific approach.New York, NY:McGraw-Hill.
  51. Lado, R.(1961).Language testing.New York, NY:McGraw-Hill.
  52. Lee, S. H.(2008).Beyond reading and proficiency assessment: The rational cloze procedure as stimulus for integrated reading, writing, and vocabulary instruction and teacher-student interaction in ESL.System,36,642-660.
  53. Lofgren, H.(1969).Measuring proficiency in the German language: A study of pupils in grade 7 (Didakometry No. 25).Malmo, Sweden:School of Education.
  54. Lonigan, C. J.,Hooe, E. S.,David, C. F.,Kistner, J. A.(1999).Positive and negative affectivity in children: Confirmatory factor analysis of a two-factor model and its relation to symptoms of anxiety and depression.Journal of Consulting and Clinical Psychology,67,374-386.
  55. MacCallum, R. C.,Roznowski, M.,Necowitz, L. B.(1992).Model modifications in covariance structure analysis: The problem of capitalization on chance.Psychological Bulletin,111,490-504.
  56. Marcoulides, G. A.(Ed.)(2001).New developments and techniques in structural equation modeling.Mahwah, NJ:Lawrence Erlbaum.
  57. Meade, A. W.,Kroustalis, C. M.(2006).Problems with item parceling for confirmatory factor analytic tests of measurement invariance.Organizational Research Methods,9(3),369-403.
  58. Muthén, L.,Muthén, B.(1998).Mplus user's guide.Los Angeles, CA:Autho.
  59. Ohnmacht, F. W.,Weaver, W. W.,Kohler, E. T.(1970).Cloze and closure: A factorial study.The Journal of Psychology,74,205-217.
  60. Oller, J. W. Jr.(1973).Cloze tests of second language proficiency and what they measure.Language Learning,23,105-118.
  61. Oller, J. W., Jr.(Ed.)(1983).Issues in language testing research.Rowley, MA:Newbury House Publishers.
  62. Oller, J. W., Jr.(Ed.),Perkins, K.(Ed.)(1980).Research in language testing.Rowley, MA:Newbury House.
  63. Oller, J. W., Jr.(Ed.),Perkins, K.(Ed.)(1980).Research in language testing.Rowley, MA:Newbury House Publishers.
  64. Oller, J. W.,Conrad, C. A.(1971).The cloze techniques and ESL proficiency.Language Learning,21,183-195.
  65. Pimsleur, P.,Stockwell, R.,Comrey, A.(1962).Foreign language learning ability.Journal of Educational Psychology,53,15-26.
  66. Rapp, D. N.,van den Broek, P.,McMaster, K. L.,Kendeou, P.,Espin, C. A.(2007).Higher-order comprehension processes in struggling readers: A perspective for research and intervention.Scientific Studies of Reading,11(4),289-312.
  67. Saito, Y.(2003).Investigating the construct validity of the cloze section in the Examination for the Certificate of Proficiency in English.Spaan fellow working papers in second or foreign language assessment,Ann Arbor, MI:
  68. Schuster, R. M.,Hammitt, W. E.,Moore, D.(2003).A theoretical model to measure the appraisal and coping response to hassles in outdoor recreation setting.Leisure Sciences,25,277-299.
  69. Spolsky, B.,Bengt, S. M.,Sako, E. W.,Aterburn, C.(1968).Preliminary studies in the development of techniques for testing overall second language proficiency.language learning,18(s3),79-103.
  70. Taboada, M. T.(2004).Building coherence and cohesion.Amsterdam:Benjamins.
  71. Weaver, W. W.,Kingston, A. J.(1963).A factor analysis of the cloze procedure and other measures of reading and language ability.The Journal of Communication,13(4),252-261.
  72. Williams, J. M.,Colomb, G. G.(2010).Style: Lessons in clarity and grace.Boston, MA:Pearson Education, Inc..
  73. Witte, S. P.,Faigley, L.(1981).Coherence, cohesion, and writing quality.College Composition and Communication,32,198-204.
  74. Yamashita, J.(2003).Processes of taking a gap-filling test: Comparison of skilled and less skilled EFL readers.Language Testing,20(3),267-293.
  75. Yu, C-Y.(2002).Los Angeles, CA.,University of California, Los Angeles, CA..
  76. Zarrabi, A.(1988).Tehran, Iran,Allameh Tabatabai University.
print cancel