「混合專家與學生實徵表現導向」大型教育評量標準設定之效度評估研究

Jin-Chang Hsieh

doi:10.6209/JORIES.202306_68(2).0001

山西大学，您好！

感谢采购，人社期刊库｜人社论文库，期间：2024/01/01~2024/12/31

感谢试用，科学期刊库｜科学论文库，期间：2024/10/20~2025/01/20

浏览高级检索

关闭
检索检索高级检索出版物检索
浏览期刊主题浏览核心期刊浏览期刊拼音浏览

关闭

查询符号(半角)	查询符号意义说明
空格	表示为「AND」两个查询词之交集
双引号 ( " " )	片语以双引号标示开始及结束，而且只寻找出现顺序相同的字词，例： " image process "
?	表示一个字母切截，输入两个?表两个字母，依此类推，例：输入「Appl?」，查得结果应为appl e , appl y … ( 常用于英文字查询 )
*	表示不限字母切截，由0~n. 例：输入「appl*」，查得结果应为appl e , appl es , appl y , appl ied , appl ication … ( 常用于英文字查询 )
AND、OR、NOT	布尔逻辑组合关键字，用来扩大或缩小查询范围的技巧。 (1) AND ：缩小查询范围 (2) OR ：扩大查询范围(3) NOT：排除不相关的范围

DOI 是数位物件识别码（ D igital O bject I dentifier ）的简称，
为物件在网路上的唯一识别码，可用于永久连结并引用目标物件。

使用DOI作为永久连结

每个DOI号前面加上「 http://dx.doi.org/ 」便成为永久网址。
如以DOI号为 10.5297/ser.1201.002 的文献为例，此文献的永久连结便是： http://dx.doi.org/ 10.5297/ser.1201.002 。
日后不论出版单位如何更动此文献位置，永久连结所指向的位置皆会即时更新，不再错失重要的研究。

引用含有DOI的文献

有DOI的文献在引用时皆应同时引用DOI。若使用APA、Chicago以外未规范DOI的引用格式，可引用DOI永久连结。

DOI可强化引用精确性、增强学术圈连结，并给予使用者跨平台的良好使用经验，目前在全世界已有超过五千万个对象申请DOI。如想对DOI的使用与概念有进一步了解，请参考（）。

ACI:

数据来源：Academic Citation Index，简称ACI

台湾最大的引用文献资料库，目前收录台湾与港澳地区所出版的人文学、社会学领域学术期刊之书目资料与参考文献，总期刊量超过690种，每年定期公布收录期刊的影响指数(Impact Factor)等指标给大众，并提供专家学者与学术单位实用的计量与分析功能。

五年影响指数（5-Year Impact Factor）：某一期刊前五年所出版的文章在当年度的平均被引用次数。

公式：（前五年发表论文在统计年的被引用次数）÷（前五年论文产出论文总篇数）

例如：A期刊2017年之五年影响指数

(A期刊2012-2016年发表论文在2017年的被引总次数)/(A期刊2012-2016年发表的论文总数)

什么是预刊文章？

为提供读者最前线之学术资讯，于期刊文献获同意刊登后、纸本印制完成前，率先于网路线上发表之文章即为预刊文章。预刊文章尚未有卷期、页次及出版日期资讯，但可藉由DOI号识别。 DOI号是文献的数位身份证字号，不论预刊或正式出版皆不会改变，读者可点击DOI连结，或于DOI号前面加上「 http://dx.doi.org/ 」连结到文献目前最新版本。

如何引用预刊文章？

请使用预刊文章的线上发表日期及DOI号来引用该篇文献。

引用范例(视不同引文格式规范可能有所差异)：

作者姓名。文章篇名。期刊名称。 YYYY/MM/DD线上预先发表。

doi:DOI 号

来源数据

P1 - 35

社会科学 > 教育学

DOI： 10.6209/JORIES

书目管理工具

相关连结

问题回报

全文下载

「混合专家与学生实徵表现导向」大型教育评量标准设定之效度评估研究

Assessing the Validity of Standard-Setting for an English Language Assessment With a Hybrid Expert and Empirical Performance Model

谢进昌(Jin-Chang Hsieh)

教育科学研究期刊； 68 卷 2 期 (2023 / 06 / 01) ， P1 - 35

繁体中文;英文 DOI： 10.6209/JORIES.202306_68(2).0001

英语文理解；混合专家与学生实徵表现导向效度评估；台湾学生成就长期追踪评量；标准本位大型评量；标准设定； English comprehension ； hybrid of expert and student empirical performance models ； Taiwan Assessment of Student Achievement: Longitudinal study ； standard-based large-scale assessment ； standard setting

0 个人觉得这篇文章赞

摘要 │ 参考文献 (54) │ 国际计量

摘要〈TOP〉

并列摘要〈TOP〉

Background and Purpose. The Taiwan Assessment of Student Achievement: Longitudinal Study (TASAL) was implemented to evaluate the effect of the new 12-year basic education curriculum on student performance in Taiwan. TASAL is a standards-based, large-scale assessment that aims to track the literacy growth of Taiwanese students, explore relevant factors, and collect empirical evidence to assist in the development of future curriculum guidelines. This study assessed the validity of standard-setting with a hybrid model combining expert and student empirical performance. The hybrid model exhibits multidimensional, multisource, and long-term cumulative features. The multidimensional feature provides evidence for procedural, internal, and external validity and for setting appropriate standards (Kane, 1994, 2001; Pant et al., 2009). The multisource feature indicates that the evidence of validity is derived from various sources, such as expert opinions and students' empirical performance. Finally, the long-term cumulative feature represents the process of accumulating evidence over a long period. Presenting every type of evidence in a study is challenging due to time and resource constraints. The burden placed on researchers and students should be considered. Method. 1. Sampling: In 2019, the evaluation of seventh-grade students was initiated formally in TASAL. In 2020, the same group of students, now in the eighth grade, was evaluated in TASAL. The sampling method was stratified two-stage cluster sampling. Initially, 256 junior high schools were selected to take part in the evaluation. Finally, 246 schools with a total of 2,793 students were enrolled for this project. Regarding the English test of TASAL, in 2019, 2,793 seventh-grade students took the TASAL English test. In 2020, 2,893 eighth-grade students took the test. Among the eighth-grade students, 2,554 took the English test in both years. 2. Materials: The TASAL English core competence assessment was developed through a standardized procedure, including purpose clarification, theory construction, assessment guidelines, performance level descriptor development, test item designation, test assembly, and data analysis. The TASAL English core competence assessment examines English reading comprehension according to the corresponding content in the 12-year basic education curriculum. Based on the concept of transforming verb-noun usage into cognitive processes and content knowledge, as proposed by Anderson et al. (2001), a separate set of assessment criteria and test items has been developed for the TASAL English core competence assessment to evaluate reading comprehension. In the TASAL English core competence assessment, six levels of performance descriptors was initially proposed (Hsieh, 2023). However, no corresponding test items were available for the sixth (highest) level of the assessment, because the standard-setting process still focused on the seventh-grade test items. Therefore, this study focused on the first five levels, which included acquiring linguistic fluency, locating explicitly stated information, literal comprehension, implicit comprehension, and evaluation and reflection beyond text comprehension. According to a review of the literature, various text types based on the OECD text types (2019) are used in the TASAL English core competence assessment, and these types are modified to include descriptive, introductive, transactional, expository, commentary, persuasive, narrative, and literary texts. The assessment for seventh-grade students contained 182 test items, and the assessment for eighth-grade students contained 196 test items; 84 common items were included in both assessments. The response consistency was good. The Expected A Posteriori (EAP) estimate of the items were 0.85 and 0.91 in the assessments for seventh-grade and eighth-grade students, respectively. 3. Standard-setting: This study employed the extended Angoff method (Hambleton & Plake, 1995) to establish assessment standards. A total of 15 experts from various regions in Taiwan were trained and participated in the standard-setting meeting. Among these experts, 10 were women and 5 were men, with an average teaching experience of 18.25 years. The standard-setting meeting was implemented in three rounds, and student ability and cutoff scores were estimated by weighted likelihood estimation (Warm, 1989). Statistical analyses were performed in R (R Core Team, 2022) and TAM software packages (Robitzsch et al., 2020). Result and Conclusion. Feedback was collected using a questionnaire on standard-setting. Most of the experts rated the process and outcome of the standard-setting meeting as being well above or above average. The experts agreed or strongly agreed that providing feedback and PLD procedures were helpful in establishing standards. In summary, this study provides satisfactory evidence for the procedural validity of standard-setting. This study also provides evidence for the internal validity of standard-setting. During the initial round, the standard error of cutoff scores was between 2.03 and 11.58, as reported by all experts across all levels. However, during subsequent rounds, the margin of error decreased. In general, most standard errors (relative to the measurement error of 34.64) were within an acceptable level of 0.33, which is consistent with the results of Kaftandjieva (2010, p. 104). Using the English comprehension performance of eighth-grade students as the external criteria, the use of the scores obtained from the seventh-grade assessment to set cutoff scores was effective for significantly distinguishing between different levels of achievement. A partial η^2 of .506 was obtained, indicating a large effect size, as suggested by Cohen (1988). In conclusion, this study provides evidence for the external validity of standard-setting. In summary, some valuable suggestions are provided based on the study results. For example, when evaluating changes in student performance, the regression toward the mean may be a crucial factor affecting the result of standard-setting during the implementation of vertical articulation of cutoff scores across grades. Additionally, continuously collecting evidence to support the validity of standard-setting is crucial in responding to educational policies and curriculum guidelines. Therefore, the study results indicate the importance of building ongoing proof of validity in future research.

参考文献 ( 54 ) 〈TOP〉

侯佩君, P.-C.,杜素豪, S.-H.,廖培珊, P.-S.,洪永泰, Y.-T.,章英华, Y.-H.(2008)。台湾乡镇市区类型之研究：「台湾社会变迁基本调查」第五期计画之抽样分层效果分析。调查研究─方法与应用，23，7-32。
连结：
张铭秋, M.-C.,黄瓅莹, L.-Y.,陈佳蓉, C.-J.,陈柏熹, P.-H.,曾芬兰, F.-L.(2022)。国中教育会考数学科的回冲效应初探。教育科学研究期刊，67(1)，227-254。
连结：
曾芬兰, F.-L.,林奕宏, Y.-H.,邱佳民, J.-M.(2017)。监控评分者效果的Yes/No Angoff标准设定法之效度检核：以国中教育会考数学科为例。测验学刊，64(4)，403-432。
连结：
谢进昌, J.-C.(2021)。以「补充性表现水平描述辅助自陈式测量构念」之延伸Angoff标准设定研究。教育心理学报，53(2)，307-334。
连结：
谢进昌, J.-C.,谢名娟, M.-C.,林世华, S.-H.,林陈涌, C.-Y.,陈清溪, C.-H.,谢佩蓉, P.-J.(2011)。大型资料库国小四年级自然科学习成就评量标准设定结果之效度评估。教育科学研究期刊，56(1)，1-32。
连结：

国际计量

〈TOP〉

E-mail ：

文章公开取用时，将寄通知信至您填写的信箱地址