漸進隨機選題之最大優先指數法在電腦化適性測驗之應用與選題控制成效之檢驗

Effect of the Progressive Maximum Priority Index Method on Item Selection Constraint in Computerized Adaptive Testing

10.7108/PT.201209.0349

黃宏宇(Hung-Yu Huang)

試題曝光控制 ； 電腦化適性測驗 ； 選題限制 ； 題庫安全 ； computerized adaptive testing ； item bank security ； item exposure control ； item selection constraint

測驗學刊

59卷3期（2012 / 09 / 01）

349 - 372

繁體中文

本研究旨在發展一個新的電腦化適性測驗選題方法，稱之為「漸進隨機選題之最大優先指數法」（progressive maximum priority index method, PG-MPI），以期能同時達到選題條件限制的控制、題庫安全性的維持，與提升題庫使用率。透過一系列的模擬研究，操弄的情境包含題庫大小與測驗題長，並比較PG-MPI與其他四種選題法（MPI法、PG法、最大訊息法與隨機選題法）之差異。結果發現在11種限制條件之下，MI法與PG-MPI法能提供最佳的選題限制之控制，且能完全控制最大試題曝光率於設定的標準內；當加速度參數設定較低的數值時，PG-MPI能提供尚佳的能力估計精準度；PG法與PG-MPI法能提高題庫的使用率。綜合上述發現，PG-MPI法兼顧PG法與MPI法的優點，不但能滿足所有選題限制，維持題庫安全、提升題庫使用率，更能保有不錯的能力估計精準度。最後，研究者也提出若干建議供未來研究參考。

This study aimed to develop a procedure for severely constrained item selection with improving item pool usage rate in computerized adaptive testing. The authors proposed the progressive maximum priority index method (PG-MPI) to combine the maximum priority index method (MPI) with the progressive method (PG). Through a series of simulation, independent of pool size and test length, it showed that compared with the other four item selection methods the PG-MPI is able to accommodate various non-statistical constraints, lead to fewer constraint violations, provide better pool security maintaining, and increase the item bank utility simultaneously with little loss in measurement precision when the acceleration parameter is set to lower value. Finally, study limitations are noted and suggestions for future investigations are proposed.

1. 許嘉凌、陳淑英(2007)。控管「變動長度」電腦適性測驗之試題曝光率與測驗重疊率。測驗學刊，54(2)，403-428。
   連結：
2. Barrada, J. R.,Olea, J.,Ponsoda, V,Abad, F. J.(2008).Incorporating randomness in the Fisher information for improving item-exposure control on CATs.British Journal of Mathematical and Statistical Psychology,61,493-513.
3. Bradlow, E. T.,Wainer, H.,Wang, X.(1999).A Bayesian random effects model for testlet.Psychometrika,64,153-168.
4. Chang, H.-H.,Qian, J.,Ying, Z.(2001).a-stratified multistage CAT with b-blocking.Applied Psychological Measurement,25,333-341.
5. Chang, H.-H.,Ying, Z.(1999).Alpha-stratified multistage computerized adaptive testing.Applied Psychological Measurement,23,211-222.
6. Chang, H.-H.,Ying, Z.(1996).A global information approach to computerized adaptive testing.Applied Psychological Measurement,20,213-229.
7. Chen, S.-Y.(2005).Controlling item exposure and test overlap on the fly in computerized adaptive testing.IMPS 2005 annual meeting of the Psychometric Society,Tilburg, Netherlands:
8. Chen, S.-Y.(2004).Controlling item exposure on the fly in computerized adaptive testing.annual meeting of the Taiwanese Psychological Association,Taipei, Taiwan:
9. Chen, S.-Y.,Ankenmann, R. D.(2004).Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing.Journal of Educational Measurement,41,149-174.
10. Chen, S.-Y.,Ankenmann, R. D.,Chang, H.-H.(2000).A comparison of item selection rules at the early stage of computerized adaptive testing.Applied Psychological Measurement,24,241-255.
11. Chen, S.-Y.,Ankenmann, R. D.,Spray, J. A.(2003).The relationship between item exposure and test overlap in computerized adaptive testing.Journal of Educational Measurement,40,129-145.
12. Chen, S.-Y.,Chang, H.-H.(2009).The maximum priority index method for severely constrained item selection in computerized adaptive testing.British Journal of Mathematical and Statistical Psychology,62,369-383.
13. Chen, S.-Y.,Lei, P.-W.(2005).Controlling item exposure and test overlap in computerized adaptive testing.Applied Psychological Measurement,29,204-217.
14. Davey, T.,Parshall, C. G.(1995).New algorithms for item selection and exposure control with computerized adaptive testing.annual meeting of the American Educational Research Association,San Francisco, CA:
15. Deng, H.,Ansley, T.,Chang, H. -H.(2010).Stratified and maximum information item selection procedures in computer adaptive testing.Journal of Educational Measurement,47,202-226.
16. Embretson, S. E.,Reise, S. P.(2000).Item response theory for psychologists.Mahwah, NJ:Lawrence Erlbaum Associates.
17. Fernandez-Ballesteros, R.(Ed.)(2003).Encyclopedia of psychological assessment.London, UK:Sage.
18. Hambleton, R. K.,Swaminathan, H.(1985).Item response theory: Principles and applications.Norwell, MA:Kluwer Academic Publishers.
19. Huang, H. Y.,Chen, P.-H.,Wang, W.-C.(2010).Computerized adaptive testing under item response model with hierarchical traits.annual meeting of the National Council on Measurement in Education,Denver, CO:
20. Ju, Y.(2005).Chia-Yi, Taiwan,National Chung Cheng University.
21. Kempf-Leonard, K.(Ed.)(2004).The encyclopedia of social measurement.San Diego, CA:Academic Press.
22. Kingsbury, G. G.,Zara, A. R.(1989).Procedures for selecting items for computerized adaptive tests.Applied Measurement in Education,2,359-375.
23. Lord, F. M.(Ed.),Novick, M. R.(Ed.)(1968).Statistical theories of mental test scores.Reading, MA:Addison-Wesley.
24. McBridge, J. R.,Martin, J. T.(1983).Reliability and validity of adaptive ability tests in a military setting.New horizons in testing,New York, NY:
25. Parshall, C. G.,Spray, J. A.,Kalohn, J. C.,Davey, T.(2002).Practical considerations in computer-based testing.New York, NY:Springer-Verlag.
26. Rasch, G.(1980).Probablistic models for some intelligence and attainment tests.Chicago, IL:The University of Chicago Press.
27. Revuelta, J.,Ponsoda, V.(1998).A comparison of item exposure control methods in computerized adaptive testing.Journal of Educational Measurement,35,311-327.
28. San Martín, E.,del Pino, G.,De Boeck, P.(2006).IRT model for ability-based guessing.Applied Psychological Measurement,30(3),193-203.
29. Segall, D. O.(2004).A sharing item response theory model for computerized adaptive testing.Journal of Educational and Behavioral Statistics,29,439-460.
30. Stocking, M. L.,Lewis, C.(1998).Controlling item exposure conditional on ability in computerized adaptive testing.Journal of Educational and Behavioral Statistics,23,57-75.
31. Stocking, M. L.,Lewis, C.(1995).A new method for controlling item exposure in computerized adaptive testing.Princeton, NJ:Educational Testing Service.
32. Stocking, M. L.,Swanson, L.(1993).A method for severely constrained item selection in adaptive testing.Applied Psychological Measurement,17(3),277-292.
33. Su, Y.-H.(2007).Chia-Yi, Taiwan,National Chung Cheng University.
34. Swanson, L.,Stocking, M. L.(1993).A model and heuristic for solving very large item selection problem.Applied Psychological Measurement,17(2),151-166.
35. Sympson, J. B.,Hetter, R. D.(1985).Controlling item-exposure rates in computerized adaptive testing.proceeding of the 27th annual meeting of the Military Testing Association,San Diego, CA:
36. van der Linden, W. J.(1998).Bayesian item-selection criteria for adaptive testing.Psychometrika,63,201-216.
37. van der Linden, W. J.(Ed.),Glas, C. A. W.(Ed.)(2000).Computerized adaptive testing: Theory and practice.Dordrecht, The Netherlands:Kluwer Academic Publishers.
38. van der Linden, W. J.(Ed.),Glas, C.(Ed.)(2000).Computerized adaptive testing: Theory and practice.Boston, MA:Kluwer Academic Publishers.
39. van der Linden, W. J.,Reese, L. M.(1998).A model for optimal constrained adaptive testing.Applied Psychological Measurement,22,259-270.
40. Veerkamp, W. J. J.,Berger, M. P. F.(1997).Some new item selection criteria for adaptive testing.Journal of Educational and Behavioral Statistics,22,203-226.
41. Wainer, H.(Ed.)(1990).Computerized adaptive testing: A primer.Hillsdale, NJ:Lawrence Erlbaum Associates.
42. Wang, W.-C.,Wilson, M. R.(2005).The Rasch testlet model.Applied Psychological Measurement,29,126-149.
43. 施慶麟(2007)。博士論文(博士論文)。嘉義縣，國立中正大學心理學系。