分析型評分規準應用於磨課師的設計類課程

Applying an Analytic Rubric to Design Type of Courses on MOOCs

10.6639/JCCIR.2016.0603.02

楊晰勛(Hsi-Hsun Yang)；王馨儀(Hsin-I Wang)

分析型評分規準 ； 同儕互評 ； 設計類課程 ； 大規模開放式線上課程 ； 動畫 ； Analytic Rubric ； Peer Assessment ； Design Type of Courses ； Massive Open Online Courses ； Animation

文化創意產業研究學報

6卷3期（2016 / 09 / 01）

11 - 22

繁體中文

MOOCs 的環境下產生龐大的作業量，所以部分老師使用了同儕互評機制，過去研究得知，為了讓學習者的評分更具公平性和學習診斷的意義，選擇分析型評分規準是一個有用且好用的評量工具。本研究建構一個評分規準量表可使用於MOOCs 學習平臺的動畫課程，目的是幫助學習者理解作業評分標準，並促進評分者保持客觀、理性和公正。研究對象均為選修「2D 動畫製作」並至少繳交四次作業之學習者。分析後結果得知，經過一段時間學習的互評者與專家的評分達一致性，並且評分者的評分具有高度的效度。本研究證實分析型評分規準改善了同儕互評的問題，同時也讓學習者透過評分規準中的學習向度做自我檢視，提升學習者的評分能力。

The prevalence of massive open online courses (MOOC) has forced teachers to rely on peer assessment systems to cope with the increased workload. A number of past papers have indicated that analytical rubrics are a favorable approach in maintaining score fairness and extrapolating learners' learning performance. However, few papers have examined the scoring rubrics for design type of courses on MOOCs. In this paper, we developed a scoring rubric scale for an applied animation course on a MOOCs platform, aiming to help learners understand the scoring standards for their assignments while ensuring that raters remain objective, reasonable, and fair. The data analyzed in this paper were peer assessment scores and interview outcomes. The participants were learners who selected ＂2D Animation Production＂ as their elective course and submitted at least four assignments. Analysis findings showed that the correlation between raters' scores and the experts' scores increased over time to achieve high correlation, suggesting an increased consistency between the scores produced by the raters and those produced by the experts. This paper verifies that analytic rubrics can effectively resolve the fairness problem. Learners can also use the learning dimensions contained in the rubric to perform self-assessments and improve their scoring abilities.

1. 林玫君(2010)。國小戲劇課程之實作評量研究—建立「戲劇基本能力」之評分規範。教育學刊，34，179-222。
   連結：
2. 張基成、吳明芳(2011)。網路化檔案評量環境下教學者評、學生自評與同儕互評之信效度比較。教育資料與圖書館學，49(1)，135-170。
   連結：
3. 張基成、吳炳宏(2011)。網路化學習歷程檔案同儕互評之信效度研究。課程與教學，14(2)，1-28。
   連結：
4. Lane, J.L. (2006). How to develop a rubric. Retrieved from https://www.schreyerinstitute.psu.edu/pdf/Develop_a_Rubric.pdf
5. OCR. (2016). State of the MOOC 2016: A year of massive landscape change for massive open online courses (2016). Retrieved from http://www.onlinecoursereport.com/state-of-the-mooc-2016-a-year-of-massivelandscape-change-for-massive-open-online-courses/
6. Wikipedia. (n.d.). Massive open online course. Retrieved from https://en.wikipedia.org/wiki/Massive_open _online_course
7. 中國大學MOOC（2016）。2D 動畫製作。取自http://www.icourse163.org/course/YUNTECH-1001547003#/info
8. Pretz, K. (2014). Low completion rates for MOOCs. IEEE roundup. Retrieved from http://theinstitute.ieee.org/ieee-roundup/members/achievements/lowcompletion-rates-for-moocs
9. Koller, D. (2012). What we're learning from online education. TED Talk. Retrieved from https://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education?language=zh-tw
10. Massive Open Online Courses. (2015). MOOC completion rates: The data. Retrieved from http://www.katyjordan. com/MOOCproject.html
11. Pickett, N., & Dodge, B. (2007). Rubrics for web lessons. Retrieved from http://webquest.org/sdsu/rubrics/weblessons.htm
12. Alias, M.,Masek, A.,Salleh, H.H.M.(2015).Self, peer and teacher assessments in problem based learning: Are they in agreements?.Procedia-Social and Behavioral Sciences,204,309-317.
13. Andrade, H.G.(1996).Understanding rubrics.Educational Leadership,54(4),14-17.
14. Andrade, H.G.(2000).Using rubrics to promote thinking and learning.Educational Leadership,57(5),13-18.
15. Auttawutikul, S.,Wiwitkunkasem, K.,Smith, D.R.(2014).Use of weblogs to enhance group learning and design creativity amongst students at a Thai University.Innovations in Education and Teaching International,51(4),378-388.
16. Bauer, C.,Figl, K.,Derntl, M.,Beran, P.P.,Kabicher, S.(2009).The student view on online peer reviews.ACM SIGCSE Bulletin,41(3),26-30.
17. Bouzidi, L.,Jaillet, A.(2009).Can online peer assessment be trusted?.Educational Technology & Society,12(4),257-268.
18. Brown, G.,Bull, J.,Pendlebury, M.(1997).Assessing student learning in higher education.London:Routledge.
19. Bryan, C.(Ed.),Klegg, K.(Ed.)(2006).Innovative Assessment in Higher Education.Abingdon, UK:Routledge.
20. Chen, C.H.(2010).The implementation and evaluation of a mobile self- and peer-assessment system.Computers & Education,55(1),229-236.
21. Cho, K.,Schunn, C.D.,Wilson, R.W.(2006).Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives.Journal of Educational Psychology,98(4),891-901.
22. Christiaans, H.H.C.M.(2002).Creativity as a design criterion.Creativity Research Journal,14,41-54.
23. Falchikov, N.(1995).Peer feedback marking: Developing peer assessment.Innovations in Education & Training International,32(2),175-187.
24. Falchikov, N.,Goldfinch, J.(2000).Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks.Reviews of Educational Research,70(3),287-332.
25. Hovardas, T.,Tsivitanidou, O.E.,Zacharia, Z.C.(2014).Peer versus expert feedback: An investigation of the quality of peer feedback among secondary school students.Computers & Education,71,133-152.
26. Jenkins, M.(2005).Unfulfilled promise: Formative assessment using computer-aided assessment.Learning and Teaching in Higher Education,1,67-80.
27. Kaiser, H.F.(1974).An index of factorial simplicity.Psychometrika,39(1),31-36.
28. Kaufman, J.H.,Schunn, C.D.(2011).Students' perceptions about peer assessment for writing: Their origin and impact on revision work.Instructional Science,39(3),387-406.
29. Kulkarni, C.,Wei, K.P.,Le, H.,Chia, D.,Papadopoulos, K.,Cheng, J.,Klemmer, S.R.(2013).Peer and self assessment in massive online classes.ACM Transactions on Computer-Human Interaction,20(6),1-31.
30. Lai, C.L.,Hwang, G.J.(2015).An interactive peerassessment criteria development approach to improving students' art design performance using handheld devices.Computers & Education,85,149-159.
31. Lin, S.S.J.,Liu, E.Z.F.,Yuan, S.M.(2001).Web-based peer assessment: Feedback for students with various thinking-styles.Journal of Computer Assisted Learning,17(4),420-432.
32. Lumley, T.,McNamara, T.F.(1995).Ratercharacteristics and rater bias: Implications for training.Language Testing,12(1),54-71.
33. Martin-Kniep, G.O.(2000).Becoming a better teacher: Eight innovations that work.Alexandria, VA:Association for Supervision and Curriculum Development.
34. McCollum, S.L.(1994).Performance assessment in the social studies classroom: A how-to book for teachers.Joplin, MO:Chalk Dust.
35. Niu, W.,Sternberg, R.J.(2001).Cultural influences on artistic creativity and its evaluation.International Journal of Psychology,36(4),225-241.
36. Piech, C.,Huang, J.,Chen, Z.,Do, C.,Ng, A.,Koller, D.(2013).Tuned models of peer assessment in MOOCs.Proceedings of the 6th international conference on educational data mining,Worcester, MA:
37. Popham, W. J.(2000).Modern educational measurement: Practical guidelines for educational leaders.Los Angeles, CA:Allyn and Bacon.
38. Reddy, Y.M.,Andrade, H.(2010).A review of rubric use in higher education.Assessment & Evaluation in Higher Education,35(4),435-448.
39. Rubin, R. F.,Turner, T.(2012).Student performance on and attitudes toward peer assessments on advanced pharmacy practice experience assignments.Currents in Pharmacy Teaching and Learning,4(2),113-121.
40. Sadler, P.M.,Good, E.(2006).The impact of selfand peer-grading on student learning.Educational Assessment,11(1),1-31.
41. Stiggins, R.J.(1994).Student-centered classroom assessment.New York:Merrill.
42. Suen, H.K.(2014).Peer assessment for massive open online courses (MOOCs).The International Review of Research in Open and Distributed Learning,15(3),312-327.
43. Sung, Y.T.,Lin, C.S.,Lee, C.L.,Chang, K.E.(2003).Evaluat ing proposal s for experiments: An application of web-based self-assessment and peer assessment.Teaching of Psychology,30(4),331-334.
44. Tinapple, D.,Olson, L.,Sadauskas, J.(2013).CritViz: Web-based software supporting peer critique in large creative classrooms.Bulletin of the IEEE Technical Committee on Learning Technology,15(1),29-35.
45. Topping, K.(1998).Peer assessment between students in colleges and universities.Review of educational Research,68(3),249-276.
46. Topping, K.J.(2005).Trends in peer learning.Educational Psychology,25(6),631-645.
47. Voss, B.D.(2013).,未出版
48. Wiggins, G.P.(1998).Educative assessment: Designing assessments to inform and improve student performances.San Francisco, CA:Jossey-Basse.
49. Yang, H.H.,Wang S.Y.,Chen S.C.(2016).Constructing a rubrics of peer assessment applied on online course in 2D animation production.Asian Conference on Arts and Humanities,Kobe, Japan:
50. Zhao, Y.(1998).The effects of anonymity on computermediated peer review.International Journal of Educational Telecommunications,4(4),311-345.
51. 王文中編、呂金燮編、吳毓瑩編、張郁雯編、張淑慧編(2004)。教育測驗與評量：教室學習觀點。臺北市:五南。
52. 王文科、王智弘(2001)。教育研究法。臺北市:五南。
53. 王馨儀、楊晰勛(2015)。網路同儕互評機制應用於「2D動畫製作」線上課程之研究。第十一屆臺灣數位學習發展研討會（TWELF 2015），高雄市:
54. 余民寧(2002)。教育測驗與評量：成就測驗與教學評量。臺北市:心理。
55. 吳明隆、涂金堂(2006)。SPSS 與統計應用分析。臺北市:五南。
56. 李坤崇(1999)。多元化教學評量。臺北市:心理。
57. 周新富(2007)。教育研究法。臺北市:五南。
58. 洪翠霞(2009)。美術資優班美術作品實作評量實施與應用。資優教育，111，8-17。
59. 徐雍智、蔡今中、陳明璋(2002)。數學創意類比與同儕評量及其網路案例設計之初探。師大學報：科學教育類，47(1)，1-13。
60. 張麗麗(2002)。評量改革的應許之地，虛幻或真實？—談實作評量之作業與表現規準。教育研究月刊，93，76-86。
61. 楊淑惠、林冠成、王鼎銘(2003)。數位化同儕互評機制應用於國小視覺藝術教學之研究。藝術與人文領域教學理論與實務研討會論文集，新竹市:
62. 楊馥如(2009)。從主觀到客觀：視覺藝術如何進行評量。美育，170，88-96。
63. 劉旨峰、楊國鑫、林珊如、袁賢銘(2003)。中學生與網路同儕互評之預測性研究。新竹師院學報，17，51-71。
64. 劉怡甫(2013)。與全球十萬人作同學：談MOOC 現況及其發展。評鑑雙月刊，42，41-44。
65. 歐滄和(2002)。教育測驗與評量。臺北市:心理。

1. (2024)。社會學習網絡支援創作作品同儕互評成效：生手與專家的差別。教育傳播與科技研究，134，67-84。