“a-分層”電腦適性測驗之曝光率控管

Item Exposure Control in A-Stratified Computerized Adaptive Testing

10.7108/PT.200812.0015

朱怡君(Yi-Jyun Jhu)；陳淑英(Shu-Ying Chen)

a-分層 ； 測驗安全 ； 試題曝光控管 ； 電腦適性測驗 ； a-stratified ； computerized adaptive testing ； item exposure control ； test security

測驗學刊

55卷4期（2008 / 12 / 31）

793 - 811

繁體中文

傳統電腦適性測驗的試題選取傾向選取高鑑別力(「a」)的試題，造成高a值的試題過度曝光，而低a值的試題幾乎都沒有被使用，針對這個問題，Chang 與Ying (1999)建議使用「a-分層」來改善，a-分層是在測驗初期，使用較低a 值的試題，而把較高a值的試題保留到後期施測。a-分層雖然可以改善題庫使用不均的問題，卻無法確保試題的最大曝光率在安全範圍內。 本研究嘗試將「線上控管法」加入a-分層中，以解決a-分層曝光率控管的問題。研究結果顯示，線上控管法不僅可以有效控管a-分層的最大曝光率，同時仍能維持良好的能力估計精確度，此外，無論是題庫異動或受試母群改變，線上控管法的控管成效皆不受影響，仍能有效控管試題曝光率。綜合這些特質，線上控管法應可做為「a-分層」電腦適性測驗曝光率控管的重要參考。

In the traditional computerized adaptive testing (CAT) design, items with high a-parameter values tend to be over administered while those with low values may rarely be administered. This problem could be solved by using the a-stratified design proposed by Chang & Ying (1999). In the a-stratified design, an item pool is stratified into several strata based on an ascending order of the a-parameter values; at the early stages of a test, items are administered from the stratum with lower a-parameter values, and those from the stratum with higher a-parameter values are administered during the later stages. Even though the a-stratified design can improve the utilization of the entire item pool, without tight control on item exposure rates, the maximum exposure rate observed from the the a-stratified design may be too high to ensure test security. This study attempted to control the maximum exposure rate by using online exposure control procedures. Results indicated that item exposure can be well controlled by implementing the online procedures in the a-stratified design without sacrificing much the precision of ability estimate. The efficiency of the online procedure remained when item pools or examinee populations have been changed. Thus, the online procedures would be good alternatives for controlling item exposure in the a-stratified CAT.

1. Chang, H. H.,Qian, J.,Ying, Z.(2001).A-stratified multistage computerized adaptive testing with b blocking.Applied Psychological Measurement,25,333-341.
2. Chang, H. H.,van der Linden, W. J.(2003).Optimal stratification of item pools in a-stratified computerized adaptive testing.Applied Psychological Measurement,27,262-274.
3. Chang, H. H.,Ying, Z.(1999).A-stratified multistage computerized adaptive testing.Applied Psychological Measurement,23,211-222.
4. Chen, S. Y.(2004).Controlling item exposure on the fly in computerized adaptive testing.Paper presented at the Annual Meeting of the Taiwanese Psychological Association,Taipei, Taiwan:
5. Chen, S. Y.,Ankenmann, R. D.(2004).Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing.Journal of Educational Measurement,41,149-174.
6. Hau, K. T.,Chang, H. H.(2001).Item selection in computerized adaptive testing: Should more discriminating items be used first.Journal of Educational Measurement,38,249-266.
7. Hau, K. T.,Wen, J. B.,Chang, H. H.(2002).Optimum number of strata in the a-stratified computerized adaptive testing design.Paper presented at the American Educational Research Association Conference,New Orleans, LA:
8. Leung, C. K.,Chang, H. H.,Hau, K. T.(2003).Incorporation of content balanncing requirements in stratification desings for computerized adaptive testing.Educational and Psychological Measurement,63,257-270.
9. Leung, C. K.,Chang, H. H.,Hau, K. T.(2002).Item selection in computerized adaptive testing: Improving the a-stratified design with the Sympson-Hetter algorithm.Applied Psychological Measurement,26,376-392.
10. Sympson, J. B.,Hetter, R. D.(1985).Controlling item-exposure rates in computerized adaptive testing.Proceedings of the 27th Annual Meeting of the Military Testing Association,San Diego, CA:
11. van der Linden, W. J.,Chang, H. H.(2003).Implementing content constraints in alpha_stratified adaptive testing using a shadow test approach.Applied Psychology Measurement,27,107-120.
12. van der Linden,W. J.,Veldkamp, B. P.(2004).Constraining item exposure in computerized adaptive testing with shadow tests.Journal of Educational and Behavioral Statistics,29,273-291.
13. Yi, Q.,Chang, H. H.(2003).A-stratified CAT design with content blocking.British Journal of Mathematical and Statistical Psychology,56,359-378.