题名

網路化檔案評量環境下教學者評量之信度與效度

并列篇名

Reliability and Validity of Instructor Assessment in a Web-Based Portfolio Assessment Environment

DOI

10.6173/CJSE.2012.2005.01

作者

張基成(Chi-Cheng Chang)；吳炳宏(Bing-Hong Wu)

关键词

信度 ； 效度 ； 電子化學習歷程檔案 ； 網路化學習歷程檔案 ； 網路化檔案評量 ； Reliability ； Validity ； e-Portfolio ； Web-Based Portfolio ； Web-Based Portfolio Assessment

期刊名称

科學教育學刊

卷期/出版年月

20卷5期（2012 / 10 / 01）

页次

393 - 412

内容语文

繁體中文
中文摘要

本研究探討網路化檔案評量環境下，教學者評量的信度與效度。研究對象為某高職二年級修習「計算機概論」課程的兩個班級學生，共72人。學習者使用網路化檔案評量系統進行個人檔案的製作、觀摩、自評與同儕互評，教師則透過系統檢視學習者的檔案並評量其學習表現。結果顯示：（1）網路化檔案評量準規具足夠之信度；（2）網路化檔案教學者評量有很高的評分者間信度及評分者內部信度，是可以展示足夠信度的評量方式；（3）網路化檔案評量準規具足夠之效度；（4）教學者檔案評量分數與學生的測驗成績具有高度的一致性，顯示網路化檔案教學者評量方法具良好效度。建議未來研究可針對簡報軟體、文書處理軟體、網頁製作軟體等建置的自我建構式數位化學習檔案來製定可行的檔案評量準規並檢驗其信、效度。
英文摘要

This study investigates the reliability and validity of instructor assessment in a web-portfolio assessment environment. The research target group consists of seventy-two students who take ”Introduction to Computer” course of the second grade in some vocational high school. During experimental period, the students use the Web-based portfolio assessment system for creating their portfolio, emulating peers' portfolio, and doing self-assessment and peer-assessment. The teacher uses the system for inspecting and assessing students' learning performances. The results reveal that the rubric of the Web-based instructor assessment has an adequate reliability. The web-based instructor assessment has a high reliability between assessors as well as a high internal reliability within one assessor. Therefore, it is an assessment method with high reliability. In regard to validity, the rubrics of the web-based portfolio assessment have an adequate validity. The consistence between portfolio instructor assessment score and test score is high, which means the web-based instructor assessment has a high validity. Finally, the study recommends to develop the rubric of self-constructed e-portfolio created by PowerPoint, Word and FrontPage software and to investigate its reliability and validity.
主题分类 社會科學 > 教育學
参考文献
  1. 岳修平、王郁青(2000)。電子化學習歷程檔案實施之態度研究。教育心理學報，31(2)，65-84。
    連結：
  2. Warren Community School. (2000). Electronic portfolio scoring rubric. Retrieved July 22, 2011, from http://seuss.wcs.sad40.k12.me.us/ep/rubric.htm
  3. Thompson, S. (2007). ePortfolio rubric that can be used with office live workspace. Retrieved July 22, 2011, from http://office.microsoft.com/en-us/templates/
  4. Morris, J. L. (2007). Rubric for assessing electronic portfolios. Retrieved July 22, 2011, from http://www.uvm.edu/~jmorris/rubricep.html
  5. Rcampus. (2010). iRubric: ePortfolio rubric - University of New England. Retrieved July 22, 2011, from http://www.rcampus.com/rubricshowc.cfm?code=W38X23&sp=true&
  6. Middlesex County College. (2009). Grading rubric. Retrieved July 22, 2011, from http://eportfolio.middlesexcc.edu/control.cfm/ID/3023
  7. e-Portfolio Consortium. (2003). Electronic portfolio white paper. Retrieved October 3, 2009, from https://portfolio.du.edu/portfolio/getportfoliofile?uid=147385
  8. Lorenzo, G., & Ittelson, J. (2005). An overview of ePortfolios. Retrieved on October 2, 2009, from http://net.educause.edu/ir/library/pdf/ELI3001.pdf
  9. University of Wisconsin-Stout. (2010). Evaluating your portfolio. Retrieved July 22, 2011, from http://www2.uwstout.edu/content/art/artedportfolios/evaluating/portevaluationrubric.html
  10. Vandervelde, J. M. (2008). Eportfolio (digital portfolio) rubric. Retrieved July 22, 2011, from http://www.uwstout.edu/soe/profdev/eportfoliorubric.html
  11. Rogers, G. (2003). RosE-portfolio. Retrieved July 9, 2011, from http://www.rose-hulman.edu/reps/
  12. ePortfolio Portal. (2009). A generic rubric for evaluating ePortfolios. Retrieved July 22, 2011, from http://www.danwilton.com/eportfolios/rubric.php
  13. San Diego State University. (2008). EDTEC masters professional e-portfolio. Retrieved July 22, 2011, from http://edtec.sdsu.edu/portfolio.htm
  14. Alter, J. A.,Spandle, V.(1992).NCME instructional module: Using portfolios of student work in in struction and assessment.Educational Measurement: Issues and Practice,11(1),36-44.
  15. Barker, K. C.(2006).,未出版
  16. Batson, T.(2002).The electronic portfolio boom: What's it all about?.Syllabus,16(5)
  17. Bloom, L.,Bacon, E.(1995).Using portfolios for individual learning and assessment.Teacher Education and Special Education,18(1),1-9.
  18. Bonita, W. L.,Duquence, L.(1999).Assessing teacher performance with a portfolio rubric.English Leadership Quarterly,22(2),6-11.
  19. Brickley, D.,Schwartz, A. I.,Suen, H. K.(2000).Web-based assessment: Validation of electronic portfolios.Annual Meeting of the American Educational Research Association,New Orleans, LA:
  20. Burnett, M. N.,Williams, J. M.(2009).Institutional uses of rubrics and e-portfolios: Spelman college and Rose-Hulman institute.Peer Review,11(1),24-27.
  21. Cambridge, B. L.(Ed.),Kahn, S.(Ed.),Tompkins, D. P.(Ed.),Yancey, K. B.(Ed.)(2001).Electronic portfolios-Emerging practice in student, faculty, and institutional learning.Washington, DC:American Association for Highter Education Press.
  22. Chang, C. C.(2008).Enhancing self-perceived effects using web-based portfolio assessment.Computers in Human Behavior,24(4),1753-1771.
  23. Chang, C. C.(2001).A study on the evaluation and effectiveness analysis of webbased learning portfolio (WBLP).British Journal of Educational Technology,32(4),435-458.
  24. Chang, C. C.,Tseng, K. H.(2009).Use and performances of web-based portfolio assessment.British Journal of Educational Technology,40(2),358-370.
  25. Chang, C. C.,Tseng, K. H.(2011).Using a web-based portfolio assessment system to elevate project-based learning performances.Interactive Learning Environments,19(3),211-230.
  26. Danielson, C.,Abrutyn, L.(1997).An introduction to using portfolios in the classroom.Alexandria, VA:Association for Supervision and Curriculum Development.
  27. Dennis, C.,Hardy, J.(2006).Development of a model to advance the uptake of e-portfolios for undergraduates in teacher education and registered nurse preparation: An exemplar of best practice.Proceedings of world conference on educational multimedia hypermedia and telecommunications 2006,Chesapeake, VA:
  28. Diperna, J.,Derham, C.(2007).Digital professional portfolios of preservice teaching: An initial study of score reliability and validity.International Journal of Technology and Teacher Education,15(3),363-381.
  29. Gadbury-Amyot, C. C.(2003).Kansas City, MO,University of Missouri.
  30. Gadbury-Amyot, C. C.,Kim, J.,Palm, R. L.,Mills, G. E.,Noble, E.,Overman, P. R.(2003).Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program.Journal of Dental Education,67(9),991-1002.
  31. Gelinas, A. M.(1998).Columbus, OH,Ohio State University.
  32. Goldsby, D.,Fazal, M.(2001).Now that your students have created web-based digital portfolios, how do you evaluate them?.Journal of Technology and Teacher Education,9(4),607-616.
  33. Johnson, R. L.,Fisher, S.,Willeke, M. J.,McDaniel, F., II.(2003).Portfolio assessment in a collaborative program evaluation: The reliability and validity of a family literacy portfolio.Evaluaion and Program Planning,26(4),367-377.
  34. Lenze, J.(2004).Inter-rater reliability in the evaluation of electronic portfolios: A survey of empirical research results.Proceedings of society for information technology & teacher education international conference 2004,Chesapeake, VA:
  35. Lynch, L. L.,Purnawarman, P.(2004).Electronic portfolio assessments in U. S. educational and instructional technology programs: Are they supporting teacher education?.TechTrend,48(1),50-56.
  36. Oskay, O. O.,Schallies, M.,Morgil, I.(2008).Reliability of Portfolio: A closer look at findings from recent publications.Hacettepe Unuversity Journal of Education,35,263-272.
  37. Rees, C. E.,Sheard, C. E.(2004).The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: The Nottingham experience.Medical Education,38(2),138-144.
  38. Russell, J. D.,Butcher, C.(1999).Using portfolios in educational technology courses.Journal of Technology and Teacher Education,7(4),279-289.
  39. Singh, O.,Ritzhaupt, A.(2006).Student perceptive of organizational uses of ePortfolios in higher education.Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006,Chesapeake, VA:
  40. Skawinski, S. F.,Thibodeau, S. J.(2003).A journey into portfolio assessment.The Education Forum,67(1),81-88.
  41. Sulzen, J.,Young, M.,Hannifin, R.(2008).Reliability and validity of an ecologically-grounded student teacher electronic portfolio rubric.Proceedings of society for information technology & teacher education international conference 2008,Chesapeake, VA:
  42. Sweat-Guy, R.,Buzzetto-More, N. A.(2007).A comparative analysis of common e-portfolio features and available platforms.Issues in Informing Science and Information Technology Education,5(1),327-342.
  43. Wang, C. I.,Peng, H. Y.,Lu, W. H.(2006).A framework for the assessing of intercultural communicative competence: A mixed approach.Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006,Chesapeake, VA:
  44. Wang, C. X.(2009).Comprehensive assessment of student collaboration in electronic portfolio construction: An evaluation research.TechTrends,53(1),58-66.
  45. Wang, S.(2009).E-portfolios for integrated reflection.Issues in Informing Science and Information Technology,6,449-460.
  46. Winsor, P. J.,Ellefson, B. A.(1995).Professional portfolios in teacher education: An exploration of their value and potential.Teacher Educator,31(1),68-81.
  47. 余民寧(2003)。教育測驗與評量。臺北市:心理。
  48. 邱皓政(2008)。量化研究與統計分析─SPSS中文視窗版資料分析範例解說。臺北市:五南。
  49. 張美玉(2001)。從多元智能的觀點談歷程檔案評量在教育上的應用。教育研究資訊，9(1)，32-54。
  50. 張麗麗(2002)。檔案評量信度與效度的分析─以國小寫作檔案為例。教育與心理研究，25，1-34。
  51. 郭生玉(2004)。教育測驗與評量。臺北市:精華。
  52. 陳英豪、吳裕益(1991)。測驗與評量。高雄市:復文。
  53. 歐滄和(2002)。教育測驗與評量。臺北:心理。
  54. 盧貞穎(2001)。讓學生做學習的主人─檔案評量的應用與省思。敦煌英語教學雜誌，30，15-21。
被引用次数
  1. 謝名娟(2022)。校務檔案怎麼評？校務經營檔案之信效度分析。教育心理學報，54(2)，333-355。
print cancel