「試題作答時間」在洩題偵測上的應用

Using Response Times to Identify Examinees with Item Pre-Knowledge

10.7108/PT.200912.0543

黃聖筠(Sheng-Yun Huang)；陳淑英(Shu-Ying Chen)

作答時間 ； 洩題偵測 ； 異常答題反應 ； 測驗安全 ； aberrant response patterns ； item pre-knowledge detection ； response time ； test security

測驗學刊

56卷4期（2009 / 12 / 01）

543 - 571

繁體中文

偵測受試者是否具有洩題資訊是維護測驗安全的重要任務，過去洩題的偵測主要是根據3PLM，並以l(下标 z)指標來偵測受試者是否具有洩題資訊，結果發現l(下标 z)有不錯的偵測成效。雖然如此，這些洩題的偵測僅考慮到受試者的作答反應，對於具有洩題資訊的受試者而言，其除了會答對難度超過其能力水準的試題外，還能快速作答，故作答時間應該也是洩題偵測的重要訊息。 本研究採用納入作答時間的4PLM，並以3PLM作為比較基礎，評估兩模式在各種洩題情境下的偵測成效。研究結果顯示，4PLM的偵測成效普遍優於3PLM，對低能力水準的受試者而言，其成效尤其顯著，因此，作答時間應可作為實務上洩題偵測的重要參考。

Test security can be improved when examinees with item pre-knowledge are effectively identified. To detect cheaters, the person-fit index, l(subscript z) is commonly used. When item sharing occurs, examinees would answer difficult items correctly, but easy items incorrectly. Based on the aberrant response patterns, l(subscript z) could be used to effectively identify examinees with item pre-knowledge. Nevertheless, response times could provide additional information for cheater detection because examinees would answer difficult items not only correctly but also quickly when they have item pre-knowledge. The purpose of this study is to improve the detection power of l(subscript z) by using 4PLM, where response times were considered. The performance of the 4PLM was evaluated based on that observed from 3PLM. Results indicated that 4PLM performed better than 3PLM in identifying examinees with item pre-knowledge, especially for low ability examinees. Thus, not only item responses but also response times should be considered in identifying examinees with item pre-knowledge.

1. Birnbaum, A.,F. M. Lord (Eds.),M. R. Novick (Eds.)(1968).Statistical theories of mental test scores.Reading, MA:Addison-Wesley.
2. Chen, S.(2009).Investigating response times for examinees with item pre-knowledge in computer based testing.
3. Drasgow, F.,Levine, M. V.(1986).Optimal detection of certain forms of inappropriate test scores.Applied Psychological Measurement,10,59-67.
4. Drasgow, F.,Levine, M. V.,McLaughlin, M. E.(1987).Detecting inappropriate test scores with optimal and practical appropriateness indices.Applied Psychological Measurement,11,59-79.
5. Drasgow, F.,Levine, M. V.,Williams, E. A.(1985).Appropriateness measurement with polychotomous item response models and standardized indices.British Journal of Mathematical and Statistical Psychology,38,67-86.
6. Frary, R. B.,Tideman, T. N.,Watts, T. M.(1977).Indices of cheating on multiple-choice tests.Journal of Educational and Behavioral Statistics,2,235-256.
7. Hendrawan, I.,Glas, C. A. W.,Meijer, R. R.(2005).The effect of person misfit on classification decisions.Applied Psychological Measurement,29,26-44.
8. Levine, M. V.,Rubin, D. B.(1979).Measuring the appropriateness of multiple-choice test scores.Journal of Educational Statistics,4,269-290.
9. Li, M. N. F.,Olejnik, S.(1997).The power of Rasch person-fit statistics in detecting unusual response patterns.Applied Psychological Measurement,2(1),215-231.
10. Nering, M. L.(1997).The distribution of indexes of person fit within the computerized adaptive testing environment.Applied Psychological Measurement,21,115-127.
11. Reise, S. P.,Due, A. M.(1991).The influence of test characteristics on the detection of aberrant response patterns.Applied Psychological Measurement,15,217-226.
12. Roskam, E. E.,W. J. van der Linden (Eds.),R. K. Hambleton (Eds.)(1997).Handbook of modern item response theory.New York:Springer.
13. Sotaridona, J. S.,van der Linden, W. J.,Meijer, R. R.(2006).Detecting answer copying using the kappa statistic.Applied Psychological Measurement,30,412-431.
14. Tatsuoka, K. K.(1984).Caution indices based on item response theory.Psychometrika,49,95-110.
15. Thissen, D.,D. J. Weiss (Ed.)(1983).New horizons in testing: Latent trait test theory and computerized adaptive testing.New York:Academic Press.
16. van der Linden,W. J.,Sotaridona, J. S.(2006).Detecting answer copying when the regular response process follows a known response model.Journal of Educational and Behavioral Statistics,31,283-304.
17. van Krimpen-Stoop, E. M. L. A.,Meijer, R. R.(1999).The null distribution of person-fit statistics for conventional and adaptive tests.Applied Psychological Measurement,23,327-345.
18. Wang, T.,Hanson, B. A.(2005).Development and calibration of an item response model that incorporates response time.Applied Psychological Measurement,29,323-339.
19. Wollack, J. A.,Cohen, A. S.,Serlin, R. C.(2001).Defining error rates and power for detecting answer copying.Applied Psychological Measurement,25,385-404.
20. Wright, B. D.,Stone, M. H.(1979).Best test design.Chicago, IL:MESA Press.