國小學童資訊素養檔案評量之信度研究

The Reliability of Information Literacy Portfolio Assessment in the Primary Grades

張郁雯(Yu-Wen Chang)

可靠性係數 ； 信度 ； 資訊素養 ； 檔案評量 ； 類推性係數 ； dependability coefficient ； generalizability coefficient ； information literacy ； portfolio assessment ； reliability

教育心理學報

39卷測驗與評量專刊（2008 / 02 / 01）

43 - 60

繁體中文

本研究的目的在探討國小資訊素養檔案的分數信度，透過三個面向檢視檔案分數的信度，分別是整體得分的類推性係數與可靠性係數、運用資訊能力歷程向度評分之變異數成分估計和類推性係數以及內外評分者的一致性信度。參與研究的為國小三年級和五年級各兩個班的學生，檔案設計以大六教學法為參考架構，結合語文、自然與生活科技、社會三個學習領域，每位學生在每個領域需完成2－3份作品，研究為期3個學期。研究結果顯示不分年級與學習領域，在僅有一位評分員的條件下，整體評分類推性係數多半可達.75以上，而可靠性係數僅略低於類推性係數，信度良好。歷程向度評分的變異數成分分析顯示，變異的主要來源有四，分別為學生差異、歷程向度、學生與歷程向度的交互作用，以及學生、評分者、和歷程向度的交互作用。歷程向度的類推性係數最佳值在0.5和0.6左右，仍有改善的空間。校內教師間的評分相關普遍高於教師與外部研究助理的相關，校內教師的給分多半較外部評分者寬鬆，不過兩者的差異量並不大。本研究顯示透過適當的檔案設計，任課教師能一致地評量資訊素養檔案。

This article reports on the reliability of information literacy portfolio assessment in the primary grades. Three central questions were examined: (1) how reliable are total scores of information portfolio? (2) how reliable are scores of big six skills? and (3) what are probable sources of unreliability? Do teachers who rate their own students' work systematically score differently than do outside raters? The framework of information portfolio was based on the big six skills approach. This assessment was based on a sample of approximately 70 portfolios integrated three learning areas for students in Grades 3 and 5: language arts, mathematics, and science and technology. The teachers were required to collect each student's portfolio for 3 semesters. The products of portfolios in each learning areas were rated by 3 raters: the student's classroom teacher, a teacher of the same primary school, and an external reviewer. Results indicated that most of the generalizability coefficients for total scores were above 0.75 across learning areas and semesters with a single rater. The dependability coefficients revealed a similar pattern with slightly lower size. Variance in scores of each processing dimension was attributed to individual difference, processing dimension, interaction of individual difference and dimension, and interaction of individual difference, dimension, and rater. The generalizability coefficients for the six-dimensional task were around 0.5 to 0.6. The results indicated that instructional planning based on scores of specific information skill should be questioned. In general, the correlations between classroom teachers were higher than those between classroom teachers and external raters. Classroom teachers tended to rate student portfolios higher than did external raters; however, these differences were small. This study showed that given a solidly structured portfolio, teachers can reliably rate their students' work.

1. (1997).Follett's information skills model.McHenry, IL:Follett Software.
2. American Association of School Librarians (1998). Information power: Learning and teaching principles of school library media programs. Retrieved Feburary 8, 2003, from http://www.ala.org/aasl/ipprinciples1.html
3. American Library Association (1989). American Library Association Presidential Committee on Information Literacy: Final report. Retrieved Feburary 8, 2003, from http://www.infolit.org/documents/89Report
4. 吳美美（民89）：資訊素養教育與e 世代教學。國家圖書館非同步遠距教學網站︰ http://cu.ncl.edu.tw/learn/index.php。檢索日期：民92.01.27
5. 教育部（ 民92）： 國民中小學九年一貫課程綱要。國民教育司︰ http://140.122.120.230/ejedata/kying/20031241215/index.htm。檢索日期：民92.02.08
6. Arter, J. A.,Spandel, V.(1992).Using portfolios of student work in instruction and assessment.Educational Measurement: Issues and Practice,11(1),36-44.
7. Baron, J. B.(ed.),Wolf, D. P.(ed.)(1991).Performance-based student assessment: Toward access, capacity and coherence.Chicago:National Society for the Study of Education.
8. Bridgeman, B.,Chittenden, E.,Cline, F.(1995).Characteristics of a portfolio scale for rating early literacy.Princeton, NJ:Educational Testing Service.
9. Eisenberg, M. B.,Berkowitz, R. E.(1990).Information problem-solving: The Big Six Sills approach to library and information skills instruction.Norwood, NJ:Ablex.
10. Eisenberg, M. B.,Berkowitz, R. E.(1999).Teaching information & Technology skills: The Big6 in Elementary Schools.Worthington, OH:Linworth.
11. Fourie, I.(1999).Using portfolio assessment in a module in research information skills.Education for Information,17(4),333-352.
12. Frechtling, J. A.(1991).Performance assessment: Moonstruck or the real thing?.Educational Measurement: Issues and Practice,10(4),23-25.
13. Galzer, S. M.,Brown, C. S.(1993).Portfolios and Beyond: Collaborative assessment in reading and writing.Norwood, MA:Christopher-Gordon.
14. Gentile, C.(1992).Exploring new ways for collecting school-based writing: NAEP's 1990 portfolio study.Washington, DC:Office of Educational Research and Improvement.
15. Herman, J. L.,Gearhart, M.,Baker, E. L.(1993).Assessing writing portfolios: Issues in the validity and meaning of scores.Educational Assessment,1(3),201-224.
16. Herman, J. L.,Winters, L.(1994).Portfolio research: A slim collection.Educational Leadership,52(2),48-55.
17. Irving, A.(1985).Study and information skills across the curriculum.London:Heineman.
18. Johnson, R. L.,McDaniel, F.,Willeke, M. J.(2000).Using portfolios in program evaluation: An investigation of interrater reliability.American Journal of Evaluation,21(1),65-80.
19. Joyce, M. Z.,Tallman, J. I.(1997).Making the writing and research connection with the I-search process.New York:Neal-Schuman.
20. Koretz, D.,Stecher, B.,Deibert, E.,CRESST(1993).,Los Angeles:Center for the Study of Evaluation, University of California.
21. Kuhlthau, C. C.(1987).An emerging theory of library instruction.School Library Media Quality,16(1),23-27.
22. LeMahieu, P. G.,Gitomer, D. H.,Eresh, J. T.(1995).Portfolios in Large-scale assessment: Difficult but not impossible.Educational Measurement: Issues and Practice,14(3),11-28.
23. Myers, M.,Pearson, P. D.(1996).Performance assessment and the literacy unit of the New Standards Project.Assessing Writing,3(1),5-29.
24. Shapley, K. S.,Bush, M. J.(1999).Developing a valid and reliable portfolio assessment in the primary grades: building on practical experience.Applied Measurement in Education,12(2),111-132.
25. Shavelson, R. J.,Webb, N. M.(1991).Generalizability theory: A Primer.Newbury Park, CA:Sage Publication.
26. Stripling, B. K.,Pitts, J. M.(1988).Brainstorm and blueprints: Library research as a thinking process.Englewood, Co:Libraries Unlimited.
27. Supovitz, J. A.,MacGowan III, A.,Slattery, J.(1997).Assessing agreement: An examiniation of the interrater reliability of portfolio assessment in Rochester, New York.Educational Assessment,4(3),237-259.
28. Tierney, R. J.,Carter, M. A.,Desai, L. E.(1991).Portfolio assessment in reading-writing classroom.Norwood, MA:Christopher-Gordon Publishers.
29. 李德竹(2000)。資訊素養的意義、內涵與演變。圖書與資訊學刊，35，1-25。
30. 張郁雯(2004)。運用資訊能力之檔案評量系統之發展研究。科技化測驗與能力指標評量國際學術研討會，台南:
31. 張麗麗(2002)。檔案評量信度與效度分析─以國小寫作檔案為例。教育與心理研究，25(1)，1-24。
32. 陳伯璋(1999)。九年一貫課程的理念、內涵與評析。國民中小學課程教學研討會，台北:
33. 鄒慧英(2004)。讀寫檔案的信度與評分者一致性。科技化測驗與能力指標評量國際學術研討會，台南:
34. 賴麗珍(1999)。國高中課程與資訊素養教育。資訊素養與終身學習社會國際研討會，台北:

1. 蔡政緯、張基成(2012)。以網路模糊德懷術與模糊層級分析法發展數位化學習歷程檔案之知識管理行為量表。教育資料與圖書館學，50(1)，103-133。
2. 陳毓彬、林菁(2013)。資訊素養評量之編製與學生表現：以國小二年級為例。教育資料與圖書館學，51(1)，91-129。
3. 簡邦宗,謝豐如,黃春木,曾慶玲,張祐穎(2022)。專題探究課程檔案評量之發展－以建中「專題寫作與表達」課程為例。通識教育學刊，30，301-347。
4. 林玫君(2010)。表演藝術評分規範之建立—以小學低年級「戲劇創作能力之應用」爲例。當代教育研究，18(2)，113-154。
5. 林玫君(2010)。國小戲劇課程之實作評量研究—建立「戲劇基本能力」之評分規範。教育學刊，34，179-222。
6. 謝名娟(2022)。校務檔案怎麼評？校務經營檔案之信效度分析。教育心理學報，54(2)，333-355。
7. 張郁雯(2010)。國小學童資訊素養檔案評量之發展研究。教育心理學報，41(3)，521-550。
8. 張郁雯(2018)。朗讀流暢性測驗標準文本之發展研究。教育實踐與研究，31(2)，1-24。