網路化學習歷程檔案學習者自評之信效度

Reliability and Validity of Learner Web-Based Portfolio Self-Assessment

10.6173/CJSE.2011.1902.03

張基成(Chi-Cheng Chang)；吳炳宏(Bing-Hong Wu)

自評 ； 學習歷程檔案 ； 網路化學習歷程檔案 ； 網路化檔案評量 ； Self-Assessment ； Portfolio ； Web-Based Portfolio ； Web-Based Portfolio Assessment

科學教育學刊

19卷2期（2011 / 04 / 01）

169 - 184

繁體中文

本研究探討網路化檔案學習者自評的信、效度。研究樣本為某高職修習「計算機概論」課程的72位學生。學生藉由網路化檔案評量系統進行個人檔案的製作、觀摩與自評。結果顯示：(1)學習者兩次檔案自評結果具有高度的一致性；(2)學習者自評與教學者評量結果具高度一致性且無顯著差異；(3)學習者檔案自評分數與測驗成績結果具高度一致性，顯示網路化檔案自評可以反應學習成就。簡言之，網路化檔案自評兼具信、效度。

This study examines the reliability and validity of Web-based portfolio self-assessment. The research sample consisted of seventy-two students taking an ”Introduction to Computers” course in the second grade of a vocational high school. The students used a Web-based portfolio assessment system to create their portfolio, emulate peers' portfolios, and perform self-assessment. The results demonstrate that there exists a high consistency between the two self-assessments. There exists a high consistency and no significant difference between student self-assessment and teacher assessment. The consistency between peer-assessment and test score was high, implying that the Web-based portfolio self-assessment may reflect learning achievement. In short, Web-based portfolio self-assessment has adequate reliable and validity.

1. 岳修平、王郁青(2000)。電子化學習歷程檔案實施之態度研究。教育心理學報，31(2)，65-84。
   連結：
2. Ballantyne, R.,Hughes, K.,Mylonas, A.(2002).Developing procedures for implementing peer assessment in large classes using an action research process.Assessment & Evaluation in Higher Education,27(5),427-441.
3. Bloxham, S.,West, A.(2004).Understanding the rules of the game: Marking peer assessment as a medium for developing students' conceptions of assessment.Assessment & Evaluation in Higher Education,29(6),721-733.
4. Bouzidi, L.,Jaillet, A.(2009).Can online peer assessment be trusted?.Educational Technology & Society,12(4),257-268.
5. Bryan, C.(ed.),Clegg, K.(ed.)(2006).Innovative assessment in higher education.London:Routledge.
6. Cambridge, B. L.(ed.),Kahn, S.(ed.),Tompkins, D. P. (ed.),Yancey, K. B.(ed.)(2001).Electronic portfolios: Emerging practice in student, faculty, and institutional learning.Washington, DC:American Association for Higher Education Press.
7. Chang, C.-C.(2008).Enhancing selfperceived effects using web-based portfolio assessment.Computers in Human Behavior,24(4),1753-1771.
8. Chang, C.-C.(2001).A study on the evaluation and effectiveness analysis of web-based learning portfolio (WBLP).British Journal of Educational Technology,32(4),435-458.
9. Chang, C.-C.,Tseng, K.-H.(2009).Use and performances of web-based portfolio assessment.British Journal of Educational Technology,40(2),358-370.
10. Chang, C.-C.,Tseng, K.-H.(2009).Using a web-based portfolio assessment system to elevate project-based learning performances.Interactive Learning Environments,16(2),1-20.
11. Cho, K.-S.,Schunn, C. D.,Wilson, R. W.(2006).Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives.Journal of Educational Psychology,98(4),891-901.
12. Davies, P.(2000).Computerized peer assessment.Innovations in Education and Training International,37(4),346-355.
13. Derham, C.,DiPerna, J.(2007).Digital professional portfolios of preservice teaching: An initial study of score reliability and validity.Journal of Technology and Teacher Education,15(3),363-381.
14. Falchikov, N.,Goldfinch, J.(2000).Student peer assesssment in higher education: A meta-analysis comparing peer and teacher marks.Review of Educational Research,70(3),287-322.
15. Gadbury-Amyot, C, C.,Kim, J.,Palm, R. L.,Mills, G. E.,Noble, E.,Overman, P. R.(2003).Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program.Journal of Dental Education,67(9),991-1002.
16. Gadbury-Amyot, C. C.(2003).Kansas, MO,University of Missouri Kansas.
17. Gelinas, A.-M.(1998).Columbus, OH.,Ohio State University.
18. Knowles, M. S.,Holton, E. F., III,Swanson, R. A.(2005).The adult learner: The definitive classic in adult education and human resource development.Burlington, MA:Elsevier.
19. Lenze, J.(2004).Inter-rater reliability in the evaluation of electronic portfolios: A survey of empirical research results.Proceedings of society for information technology & teacher education international conference 2004,Chesapeake, VA:
20. Lin, S. S. J.,Liu, E. Z.-F.,Yuan, S.- M.(2001).Web-based peer assessment: Feedback for students with various thinking-styles.Journal of Computer Assisted Learning,17(4),420-432.
21. Liu, E. Z. F.,Lin, S. S. J.,Chiu, C. H.,Yuan, S. M.(2001).Web-based peer review: The learner as both adapter and reviewer.IEEE Transactions on Education,44(3),246-251.
22. Luca, J.,McMahon, M.(2006).Developing multidisciplinary teams through self-assessment, supported with online tools.Proceedings of ed-media,Norfolk, VA:
23. Miller, P. J.(2003).The effect of scoring criteria specificity on peer and selfassessment.Assessment & Evaluation in Higher Education,28(4),383-394.
24. Oskay, O. O.,Schallies, M.,Morgil, I.(2008).Reliability of portfolio: A closer look at findings from recent publications.Hacettepe University Journal of Education,35,263-272.
25. Prins, F. J.,Sluijsmans, D. M. A.,Kirschner, P. A.,Strijbos, J.-W.(2005).Formative peer assessment in a CSCL environment: A case study.Assessment & Evaluation in Higher Education,30(4),417-444.
26. Russell, J. D.,Butcher, C.(1999).Using portfolios in education technology courses.Journal of Technology and Teacher Education,7(4),279-290.
27. Sadler, P. M.,Good, E.(2006).The impact of self- and peer-grading on student learning.Educational Assessment,11(1),1-31.
28. Sluijsmans, D.,Dochy, F.,Moerkerke, G.(1999).Creating a learning environment by using self-, peer-, and co-assessment.Learning Environments Research,1(3),293-319.
29. Sulzen, J.,Young, M. F.,Hannifin, R.(2008).Reliability and validity of an ecologically-grounded student teacher electronic portfolio rubric.Proceedings of society for information technology & teachereducation international conference 2008,Chesapeake, VA:
30. Sung, Y. T.,Lin, C. S.,Lee, C. L.,Chang, K. E.(2003).Evaluating proposals for experiments: An application of webbased self-assessment and peer-assessment.Teaching of Psychology,30,331-334.
31. Sung, Y.-T.,Chang, K. E.,Chiou, S. K.,Hou, H. T.(2005).The design and application of a web-based self- and peer-assessment system.Computers & Education,45(2),187-202.
32. Topping, K. J.,Smith, E. F.,Swanson, I.,Elliot, A.(2000).Formative peer assessment of academic writing between postgraduate students.Assessment & Evaluation in Higher Education,25(2),149-169.
33. Tsai, C. C.,Liang, J. C.(2009).The development of science activities via online peer assessment: The role of scientific epistemological views.Instructional Science,37(3),293-310.
34. Tsai, C. C.,Lin, S. S. J.,Yuan, S. M.(2002).Developing science activities through a networked peer assessment system.Computers & Education,38(1-3),241-252.
35. Wen, M. L.,Tsai, C. C.(2006).University students' perceptions of and attitudes toward (online) peer assessment.Higher Education,51(1),27-44.
36. Yang, Y. F.,Tsai, C. C.(2010).Conceptions of and approaches to learning through online peer assessment.Learning and Instruction,20(1),72-83.
37. 王家玲(2002)。碩士論文(碩士論文)。桃園縣，國立中央大學人力資源管理研究所。
38. 王等元(2000)。「歷程檔案」概念及其在學習者支持系統之意涵。社會教育學刊，29，249-274。
39. 余民寧(2002)。教育測驗與評量―成就測驗與教學評量。臺北市:心理。
40. 張麗麗(2002)。檔案評量信度與效度的分析―以國小寫作檔案為例。教育與心理研究，25(1)，1-34。
41. 郭怡雯(2001)。國科會大專生參與專題研究計畫報告國科會大專生參與專題研究計畫報告,臺北市:行政院國家科學委員會。
42. 陳英豪、吳裕益(1990)。測驗與評量。高雄市:復文書局。
43. 陳嘉鴻(2002)。碩士論文(碩士論文)。高雄市，國立高雄師範大學資訊教育研究所。
44. 劉旨峰、林珊如、袁賢銘(2002)。網路化學習歷程檔案評量系統、學習成效分析及未來展望。第三屆電子化企業經營管理理論暨實務研討會，彰化縣: