Effects of Gaps-Minimizing Approaches on the Raw-to- Scale Score Conversions When Forms Vary in Difficulty

量尺分數間距縮小法在題本難易不同時效果之探討

章舜雯(Shun-Wen Chang)

分數間距 ； 原始至量尺分數轉換 ； 基本學力測驗 ； 量尺分數 ； 題本難易 ； BCTEST ； form difficulty ； gaps ； raw-to-scale score conversions ； scale score

教育心理學報

39卷測驗與評量專刊（2008 / 02 / 01）

151 - 174

英文

本研究針對在題本難易不同的情況下，探討使用量尺分數間距縮小法，縮小由原始分數轉換至量尺分數後所產生的間距之效果。本研究透過三參數extended beta-binomial模式，模擬產生一如民國90－92年期間基本學力測驗五個學科的考生分數分配，評鑑「無調整法」、「同等平均數法」、「不同等平均數法」、以及「不同等平均數及標準差法」這四種方法在不同測驗年間的表現。基本學力測驗為一標準化測驗，各學科在這幾年內的題本難易度略有不同。本研究設定將量尺高分一端縮小分數間距至3，4，5分，評鑑的準則包含量尺分數描述統計值、測驗信度、整體測量誤差以及在不同真分數下的測量標準誤大小，還有調整分數間距後對低分一端所更動的量尺點數。研究結果指出，題本難易會影響各間距縮小法的表現，其中沒有任何一種方法能達到縮小量尺高分一端分數間距的目標，而卻不會帶來任何負面的效果。調整量尺分數的間距對於較容易的題本所產生的影響大於較困難的題本；題本難度增加後，各間距縮小法之影響性也隨之降低。整體而言，「不同等平均數及標準差法」似乎是最好的選擇。本研究的結果應能促進對分數間距議題的了解，並也喚起了對題本難易度在建立量尺時所扮演的角色的注意。

This study explored the various adjusting procedures for minimizing the size of gaps resulting from the raw-to- scale score conversions under test forms that varied in difficulty. The no adjustment, the fixed mean, the varying mean, and the varying mean/SD approaches were compared using the data simulated based on the three-parameter extended beta-binomial model for the five tests in the Basic Competence Test (or BCTEST) administered from 2001 to 2003. The BCTEST is a national standardized assessment in Taiwan and the forms of each of its tests varied slightly in difficulty over these years. The desired gap sizes were set at 3, 4, and 5 scale score points at the high end of the scale. The criteria for comparing the adjusting approaches over the years were by means of the summary statistics, reliability, overall SEM, SEMs by true score in proportioncorrect score units, and the number of scale score points changed due to the truncation as well. The results showed that test form difficulty affected the performance of the various adjusting procedures to some extent and no one method could accomplish the goal of reducing the gap sizes at the upper end without negatively affecting the other scale score attributes. Imposing adjustments on the gaps at the high end of the scale would exert more effects on the easier forms than on the harder forms. The impact due to adjustments decreased as the forms increased in difficulty. Overall, the varying mean/SD strategy was judged the most preferable. Findings from this research have fostered the understanding of the gaps issue and have raised greater awareness of the role that test form difficulty plays in establishing the score scales.

1. Chang, S. W.(2007).Comparisons of score transformation methods for the BCTEST using real and simulated data.Chinese Journal of Psychology,49(2),105-135.
   連結：
2. Brennan, R. L.(ed.)(1989).Methodology used in scaling the ACT Assessment and P-ACT+.Iowa City, IA:American College Testing Program.
3. Carlin, J. B.,Rubin, D. B.(1991).Summarizing multiple-choice tests using three informative statistics.Psychological Bulletin,110,338-349.
4. Chang, S. W.(2005).Explorations of adjusting procedures for minimizing gaps in the raw-to-scale score conversions for the BCTEST.the annual meeting of the National Council on Measurement in Education,Montreal:
5. Chang, S. W.(2006).Methods in scaling the Basic Competence Test.Educational and Psychological Measurement,66,907-929.
6. Dorans, N. J.(2002).College Board Research Report No. 2002-11College Board Research Report No. 2002-11,New York:The College Board.
7. Dorans, N. J.(2002).ETS Research Report RR-02-04ETS Research Report RR-02-04,New York:The College Board.
8. Keats, J. A.,Lord, F. M.(1962).A theoretical distribution for mental test scores.Psychometrika,27,59-72.
9. Kolen, M. J.(1988).Defining score scales in relation to measurement error.Journal of Educational Measurement,25,97-110.
10. Kolen, M. J.,Hanson, B. A.,Brennan, R. L.(1992).Conditional standard errors of measurement for scale scores.Journal of Educational Measurement,29,285-307.
11. Linn, R. L.(ed.)(1989).Educational measurement.New York:American Council on Education, and Macmillan.
12. Lord, F. M.,Novick, M. R.(1968).Statistical theories of mental test scores.Reading, MA:Addison-Wesley.

1. (2009)。國中基測量尺系統相關議題之探討。中等教育，60(1)，106-119。