電腦適性測驗題目曝光率之模擬研究

The Simulating Study of the Item Exposure Rate in Computerized Adaptive Tests

10.6773/JRMS.200606.0059

謝友詩(Yu-Shin Hsieh)；劉湘川(Hsiang-Chuan Liu)；郭伯臣(Bor-Chen Kuo)

電腦適性測驗 ； 題目曝光率 ； 題目反應理論 ； 選題法 ； computerized adaptive testing ； item exposure rate ； item response theory ； selection criterion

測驗統計年刊

14期_上（2006 / 06 / 01）

59 - 74

繁體中文

電腦適性測驗在實際實施後，最受注目的議題便是題目的過度曝光，題目的過度曝光表示大部分的受測者施測過此題目，當受測者重新施測，則容易施測到相同的題目使得測驗的安全性與公平性產生危機。 本研究採用五種選題方法，分別為最接近偏移難度法、區間式最大訊息法、KL訊息法、鄰近法、與考慮b參數的a分層法，分別討論在不同題庫樣式下對於曝光率均勻度與能力估計誤差的表現，結果發現，各適性選題法依曝光率均勻度與能力估計精準度的表現上可分為三大類：一是有較高估計精準度的區間式最大訊息法與KL訊息法，一是有較均勻題目曝光的最接近偏移難度法與考慮b參數的a分層法，一是對均勻題目曝光率與估計精準度較折衷的鄰近法。

For operational computerized adaptive tests, the most important issue is the overexposure item rates. The item having overexposure rate means most of the examinees tested it. When the examinees retest, they tend to test the same items, which leads to serious test security and equity risks. In this study, discuss the effects of the five item selection criterions-minimum offset difficulty, maximum interval information, KL information, NN criterion, and STR-B-were compared with respect to the precision of the trait estimation and the effect of the item usage at the same item banks. In the result, by the exposure rate and the precision the selection criterions could separate to three groups: maximum interval information and KL information criterions which having more precision of estimation; minimum offset difficulty criterion and STR-B which having more uniform exposure rates; NN criterion which balancing the estimation precision and effective item usage.

1. Baker, F. B.(1990).Some observations on the metric of PC-BILOG results.Applied psychological measurement,14,139-150.
2. Birnbaum, A.(1968).Statistical theories of mental test scores.Addison-Wesley:Reading, Mass.
3. Chang, H. H.,Qian, J.,,Ying, Z.(2001).a-stratified multistage CAT with b-blocking.Applied Psychological Measurement,25,333-341.
4. Chang, H. H.,Ying, Z.(1999).a-stratified multistage computerized adaptive testing.Applied Psychological Measurement,23(3),211-222.
5. Chang, H. H.,Ying, Z.(1996).A global information approach to computerized adaptive testing.Applied Psychological Measurement,20(3),231-229.
6. Chen, S. Y.,Ankenmann, D.,Chang, H. H.(2000).A comparison of item selection rules at the early stages of computerized adaptive testing.Applied Psychological Measurement,24(3),241-255.
7. Cheng, P. E.,Liou, M.(2003).Computerized adaptive testing using the nearest neighbors criterion.Applied Psychological Measurement,24,257-265.
8. Cover, T. M.,Thomas, J. A.(1991).Elements of information theory.New York:Wiley.
9. Drasgow, F.(1989).An evaluation of marginal maximum likelihood estimation for the two-parameter logistic model.Applied psychological measurement,13,77-90.
10. Hung, P. H.(1988).University of Minnesota, Minnesota..
11. Kullback, S.(1959).Information theory and statistics.New York:Wiley.
12. Lord, F. M.(1977).A broad-range tailored test of verbal ability.Applied Psychological Measurement,1(1),95-100.
13. Mislevy, R. J.,Stocking, M. L.(1989).A consumer's Guide to LOGIST and BILOG.Applied Psychological Measurement,13(1),57-75.
14. Skaggs, G.,Stevenson, J.(1989).A comparison of pseudobayesian and joint maximum likelihood procedures for estimating item parameters in the three-parameter IRT model.Applied Psychological Measurement,13(4),391-402.
15. Stocking, M. L.(1994).Three practical issues for modern adaptive testing item pools. Educational Testing Service, Princeton, N. J.
16. Stone, C. A.(1992).Recovery of marginal maximum likelihood estimates in the two parameter logistic response model: An evaluation of MULTILOG.Applied psychological measurement,16,1-16.
17. Sympson, J. B.,Hetter, R. D.(1985).Controlling item exposure rates in computerized adaptive testing.In Proceedings of the 27th annual meeting of the Military Testing Association,San Diego, CA:
18. Urry, B. W.(1977).Tailored testing: A successful application of latent trait theory.Journal of educational measurement,14,181-196.
19. Veerkamp, W. J. J.,Berger, M. P. F.(1997).Some new item selection criteria for adaptive testing.Journal of Educational and Behavioral Statistics,22,203-226.
20. Wang, T.(1997).meeting of the American Educational Research Association.Chicago.
21. Weiss, D. J.(1973).The stratified adaptive computerized ability test (Research Report RR-73-3).Princeton, NJ:Educational Testing Service.
22. Yi , Q.,Chang, H. H.(2003).Journal of Mathematical and Statistical Psychology.
23. 王寶墉(1995)。現代測驗理論。台北市:心理。
24. 李茂能(2000)。中文電腦化適性測驗系統之應用與評鑑。台北市:文景。
25. 洪碧霞、吳裕益、吳鐵雄、陳英豪(1992)。國科會計劃國科會計劃,未出版
26. 陳俊宏(2004)。台中市，國立台中師範學院。
27. 陳新豐(1999)。台南市，國立臺南師範學院。
28. 陳麗如(1998)。台北市，國立臺灣師範大學。

1. 陳世銘、郭伯臣、張俊欽(2008)。以a-鄰近法為選題策略之電腦化適性測驗系統。測驗統計年刊，16(上)，33-54。
2. 錢永財、陳曉竹、郭伯臣(2009)。以a—鄰近法爲選題策略之電腦化適性測驗模擬研究。測驗學刊，56(4)，605-638。
3. 王榮照(2014)。模擬研究適合的實驗設計。運動教練科學，33，67-77。