網路化檔案評量環境下教學者評、學生自評與同儕互評之信效度比較

Comparisons of Reliability and Validity among Instructor Assessment, Student Self-Assessment and Peer-Assessment under Web-Based Portfolio Assessment Environment

10.6120/JoEMLS.201109_49(1).0386.RS.CM

張基成(Chi-Cheng Chang)；吳明芳(Ming-Fun Wu)

網路化學習歷程檔案 ； 網路化檔案評量 ； 自評 ； 同儕互評 ； Web-based portfolio ； Web-based portfolio assessment ； Self-assessment ； Peer-assessment

教育資料與圖書館學

49卷1期（2011 / 09 / 01）

135 - 170

繁體中文

本研究比較網路化檔案評量環境下教學者評、學生自評與同儕互評的信效度。研究樣本為某高職二年級修習「計算機應用」課程的72位學生。學生使用網路化檔案評量系統進行檔案的製作、觀摩、自評與同儕互評，教師透過系統檢視學生的檔案並評量其學習表現。在三種評量結果之間差異上，教學者評、學生自評、同儕互評三種評量結果有顯著差異，其中教學者評量最為嚴格、其次為學生自評，同儕互評最為寬鬆。在三種評量結果之間一致性上，僅自評與教師評之間具一致性。自評與同儕互評之間、同儕互評與教學者評之間，都不具一致性。在三種評量結果與測驗成績一致性上，教學者評分結果與測驗成績具高度一致性，學生自評與測驗成績亦具高度一致性；而同儕互評結果與測驗成績一致性低。

This study compares the reliable and validity among instructor assessment, student self-assessment and peer-assessment under the Web-based portfolio assessment environment. The research samples consist of 72 students who take ”Computer Application” course of the second grade in some vocational high school. The students use the Web-based portfolio assessment system for creating their portfolio, emulating peers' portfolio, and performing self-assessment and peer-assessment. The teacher uses the system for reviewing and assessing students' learning performances. In regard to the difference among the three assessment approaches, there exists a significant difference among the three approaches. Instructor assessment is most rigorous, while peer-assessment is most slack. In regard to the correlation among the three assessment approaches, there exists a significant consistency between self-assessment and instructor assessment. There exist inconsistencies between self-assessment and peer-assessment as well as peer-assessment and instructor assessment. In regard to the correlation among the three assessment results and test score, there exist a high consistency between instructor assessment and test score. As well, there exist a high consistency between self-assessment and test score. However, the consistency between peer-assessment and test score is inadequate.

1. High Plains Regional Technology in Education Consortium (2005). Educational technology MA online program rubric: EPortfolio. Retrieved February 22, 2010, from http://www.msarreola.com/Master%20Canvas/Assessment%20&%20Evaluation_files/eport_rubric.pdf
2. 鄒慧英[Tsou, Hui-Ying]（2000）。檔案於國語科教學評量之應用[Dangan yu guoyuke jiaoxue pingliang zhi yingyong]。上網日期：2009年8月7日[Retrieved August 7, 2009]，檢自[from]：http://www.nknu.edu.tw/~edu/item/item4-article.file/item4-article19.htm
3. Milford School District, (2009). Information, communication, technology (ICT) portfolio assessment. Retrieved February 22, 2010, from http://www.milfordschools.net/mms/documents/ePortfolio%20Rubric%202009.pdf
4. Anderson, L.,Krathwohl, D.(2001).A taxonomy for learning teaching and assessing: A revision of Bloom's taxonomy of educational objectives.New York:Wesley Longman.
5. Ballantyne, R.,Hughes, K.,Mylonas, A.(2002).Developing procedures for implementing peer assessment in large classes using an action research process.Assessment & Evaluation in Higher Education,27(5),427-441.
6. Barrett, H.(2006).Using electronic portfolios for classroom assessment.Connected Newsletter,13(2),4-6.
7. Bloxham, S.,West, A.(2004).Understanding the rules of the game: Marking peer assessment as a medium for developing students' conceptions of assessment.Assessment & Evaluation in Higher Education,29(6),721-733.
8. Burch, C. B.(1999).Inside the portfolio experience: The student's perspective.English Education,32(1),34-49.
9. Chang, C. C.(2008).Enhancing self-perceived effects using web-based portfolio assessment.Computers in Human Behavior,24(3),1753-1771.
10. Chang, C. C.,Tseng, K. H.(2009).Using a web-based portfolio assessment system to elevate project-based learning performances.Interactive Learning Environments,16(2),25-37.
11. Chang, C. C.,Tseng, K. H.(2009).Use and performances of web-based portfolio assessment.British Journal of Educational Technology,40(2),358-370.
12. Chang, C. C.,Tseng, K. H.,Yueh, H. P.,Lin, W. C.(2011).Consideration factors and adoption of type, tabulation and framework for creating e-portfolios.Computers & Education,56(2),452-465.
13. Chen, C. H.(2010).The implementation and evaluation of a mobile self- and peer-assessment system.Computers & Education,55(1),229-236.
14. Cho, K.,Schunn, C.,Wilson, R.(2006).Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives.Journal of Educational Psychology,98(4),891-901.
15. Davies, P.(2000).Computerized peer assessment.Innovations in Education and Training International,37(4),346-355.
16. Derham, C.,Diperna, J.(2007).Digital professional portfolios of preservice teaching: An initial study of score reliability and validity.International Journal of Technology and Teacher Education,15(3),363-381.
17. Falchikov, N.,Goldfinch, J.(2000).Student peer assessment in higher education: A metaanalysis comparing peer and teacher marks.Review of Educational Research,70(3),287-322.
18. Gadbury-Amyot, C.(2003).Kansas City,University of Missouri-Kansas City.
19. Gadbury-Amyot, C.,Kim, J.,Palm R. L.,Mills, E.,Noble, E.,Overman, P.(2003).Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program.Journal of Dental Education,67(9),991-1002.
20. Gelinas, A. M.(1998).Columbus, Ohio,Ohio State University.
21. Jenkins, M.(2004).Unfulfilled promise: Formative assessment using computer-aided assessment.Learning and Teaching in Higher Education,1,67-80.
22. Johnson, R. L.,McDaniel, F.,Willeke, M. J.(2000).Using portfolios in program evaluation:An investigation of interrater reliability.American Journal of Evaluation,21(1),65-81.
23. Knowles, M. S.,Holton III, E. F.,Swanson, R. A.(2005).The adult learner.Burlington, MA:Elsevier.
24. Lin, S. J.,Liu, Z. F.,Yuan, S. M.(2001).Web-based peer assessment: Attitude and achievement.IEEE Transactions on Education,44(2),211-223.
25. Lin, S. J.,Liu, Z. F.,Yuan, S. M.(2001).Web-based peer assessment: Feedback for students with various thinking-styles.Journal of Computer Assisted Learning,17(4),420-432.
26. Liu, Z. F.(2002).Hsinchu,National Chiao Tung University.
27. Liu, Z. F.,Lin, S. J.,Chiu, C. H.,Yuan, S. M.(2001).Web-based peer review: The learner as both adapter and reviewer.IEEE Transactions on Education,44(3),246-251.
28. Miller, P. J.(2003).The effect of scoring criteria specificity on peer and self-assessment.Assessment & Evaluation in Higher Education,28(4),383-394.
29. Oskay, O.,Schallies, M.,Morgil, I.(2008).A closer look at findings from recent publication.H. U. Journal of Education,35,263-272.
30. Price, M.,O''Donovan, B.(2006).Improving performance through enhancing student understanding of criteria and feedback.Innovative assessment in higher education,London, UK:
31. Prins, F. J.,Sluijsmans, D. M. A.,Kirschner, P. A.,Strijbos, J. W.(2005).Formative peer assessment in a CSCL environment: A case study.Assessment & Evaluation in Higher Education,30(4),417-444.
32. Reckase, M. A.(2002).Portfolio define.Workshop of Portfolio Assessment,Taipei, Taiwan.:
33. Russell, J. D.,Butcher, C.(1999).Using portfolios in educational technology courses.Journal of Technology and Teacher Education,7(4),279-289.
34. Sadler, P.,Good, E.(2006).The impact of self- and peer-grading on student learning.Educational Assessment,11(1),1-31.
35. Sulzen, J.,Young, M.,Hannifin, R.(2008).Reliability and validity of an ecologicallygrounded student teacher electronic portfolio rubric.Proceedings of Society for Information Technology & Teacher Education International Conference,Chesapeake, VA:
36. Sung, Y. T.,Chang, K. E.,Chiou, S. K.,Hou, H. T.(2005).The design and application of a web-based self and peer-assessment system.Computers & Education,45,187-202.
37. Sung, Y. T.,Lin, C. S.,Lee, C. L.,Chang, K. E.(2003).Evaluating proposals for experiments: An application of web-based self-assessment and peer-assessment.Teaching of Psychology,30(4),331-334.
38. Sweat-Guy, R.,Buzzetto-More, N.(2007).A comparative analysis of common e-portfolio platforms and available features.Issues in Informing Science and Information Technology Education,5(1),327-342.
39. Tabachnick, B. G.,Fidell, L. S.(2007).Using multivariate statistics.Boston, MA:Allyn & Bacon.
40. Topping, K. J.,Smith, E. F.,Swanson, I.,Elliot, A.(2000).Formative peer assessment of academic writing between postgraduate students.Assessment & Evaluation in Higher Education,25(2),149-169.
41. Tsai, C. C.,Liang, J. C.(2009).The development of science activities via on-line peer assessment: The role of scientific epistemological views.Instructional Science,37,293-310.
42. Tsai, C. C.,Lin, S. J.,Yuan, S. M.(2002).Developing science activities through a networked peer assessment system.Computers & Education,38,241-252.
43. Tseng, S. C,Tsai, C. C.(2007).On-line peer assessment and the role of the peer feedback: A study of high school computer course.Computers & Education,49(4),1161-1174.
44. Wen, L. M. C.,Tsai, C. C.(2006).University students' perceptions of and attitudes toward (Online) peer assessment.Higher Education,51,27-44.
45. Yang, Y. F.,Tsai, C. C.(2010).Conceptions of and approaches to learning through online peer assessment.Learning and Instruction,20,72-83.
46. 王保進(2006)。中文視窗版SPSS 與行為科學研究。台北市=Taipei:心理出版社=Psychological Publishing Co., Ltd.。
47. 余民寧(2003)。教育測驗與評量：成就測驗與教學評量。台北市=Taipei:心理出版社=Psychological Publishing Co., Ltd.。
48. 吳明隆、涂金堂(2008)。SPSS與統計應用分析。台北市=Taipei:五南=Wu-Nan。
49. 卓宜青(2001)。新竹市=Hsinchu，國立交通大學資訊科學系=Department of Computer and Information Science, National Chiao Tung University。
50. 邱皓政(2008)。量化研究與統計分析：SPSS中文視窗版資料分析範例解說。台北市=Taipei:五南=Wu-Nan。
51. 張振成(1997)。教學評量的新趨勢：實作評量與檔案評量。中等教育，48(6)，90-94。
52. 曾吉雄(1998)。新式評量：檔案評量。教育資料文摘，48(1)，133-137。
53. 楊國鑫(2002)。新竹市=Hsinchu，國立交通大學資訊科學系=Department of Computer and Information Science, National Chiao Tung University。
54. 劉旨峰、林珊如、袁賢銘(2002)。網路化學習歷程檔案之學習成效分析及未來展望。第三屆電子化企業經營管理理論暨實務研討會，彰化縣=Changhua:
55. 劉勝鈺(2003)。新竹市=Hsinchu，國立交通大學網路學習學程碩士班=Degree Program of CS College, National Chiao Tung University。

1. 黃芷翎(2022)。探討通識課程導入探究實作之學習成效：以仿生材料為例。通識學刊：理念與實務，10(2)，41-78。
2. 連興隆,張雅淨(2020)。運用BookRoll電子書學習系統探討學習投入與學習成效之關係。國立臺灣科技大學人文社會學報，16(4)，383-409。
3. 劉光夏(2021)。跨越主觀：網路同儕互評應用於大學藝術設計課堂之實踐研究。設計學報，26(2)，69-92。
4. 楊晰勛、王馨儀(2016)。分析型評分規準應用於磨課師的設計類課程。文化創意產業研究學報，6(3)，11-22。
5. (2022)。同儕互評可行嗎？同儕互評在儲訓校長班之信效度研究。教育與心理研究，45(1)，29-52。