题名

數位學習實驗研究品質評估與現況分析：以行動學習為例

并列篇名

Quality Assessment and Situational Analysis of Experimental E-Learning Designs: A Case Study of Mobile Learning

DOI

10.6209/JORIES.2017.62(2).02

作者

李漢岳(Han-Yueh Lee)；楊介銘(Je-Ming Yang)；宋曜廷(Yao-Ting Sung)

关键词

行動學習 ； 研究品質 ； 實驗設計 ； experimental design ； mobile learning ； research quality

期刊名称

教育科學研究期刊

卷期/出版年月

62卷2期（2017 / 06 / 01）

页次

31 - 60

内容语文

繁體中文
中文摘要

由於近年來行動裝置與教育應用軟體的快速發展，探討行動裝置輔助學習效果的實驗研究大量增加。然而，目前尚缺乏以評估實驗研究品質為主題所進行的回顧性研究。因此，本研究的目的即為廣泛蒐集ERIC 和SSCI 資料庫自2003 年至2013 年間，197 篇以行動裝置輔助學習的實驗研究，並探討其在實驗設計嚴謹度、統計考驗之過程與結果的正確性，以及成效評估工具的品質等議題。研究結果顯示，實驗設計以準實驗研究占最多數（61%），但有四分之一的準實驗研究未考量實驗組與控制組之起點能力均等性問題。統計結果效度部分則發現，半數以上研究未符合統計基本假設，七成研究所採用之樣本數偏低，使其統計考驗力低落，效果值估計情形未臻精準。成效評估工具品質部分，則發現半數研究者未提供相關的測驗信度與效度資訊。最後，本研究根據研究結果與討論，對未來研究者提出相關建議。
英文摘要

Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning-based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.
主题分类 社會科學 > 教育學
参考文献
  1. 張菀珍、葉榮木(2014)。應用多媒體行動學習系統輔助大學生情緒管理學習成效與評量之探究。教育科學研究期刊，59(4)，99-136。
    連結：
  2. U.S. Department of Education. (2002). Strategic plan 2002-2007. Retrieved from http://www2.ed.gov/about/reports/strat/plan2002-07/index.html
  3. What Works Clearinghouse. (2014). WWC procedures and standards handbook version 3.0. Retrieved from http://ies.ed.gov/ncee/wwc/Handbooks
  4. Ardito, C.,Costabile, M. F.,de Angeli, A.,Lanzilotti, R.(2012).Enriching archaeological parks with contextual sounds and mobile technology.ACM Transactions on Computer-Human Interaction,19(4),1-30.
  5. Aron, A.,Aron, E. N.(1999).Statistics for psychology.Upper Saddle River, NJ:Prentice-Hall.
  6. Basoglu, E. B.,Akdemir, O.(2010).A comparison of undergraduate students English vocabulary learning: Using mobile phones and flash cards.Turkish Online Journal of Educational Technology,9(3),1-7.
  7. Bebell, D.,O'Dwyer, L. M.(2010).Educational outcomes and research from 1:1 computing settings.Journal of Technology, Learning, and Assessment,9(1),5-15.
  8. Bernard, R. M.,Borokhovski, E.,Schmid, R. F.,Tamim, R. M.,Abrami, P. C.(2014).A meta-analysis of blended learning and technology use in higher education: From the general to the applied.Journal of Computing in Higher Education,26(1),87-122.
  9. Billings, E. S.,Mathison, C.(2012).I get to use an iPod in school? Using technology-based advance organizers to support the academic success of English learners.Journal of Science Education and Technology,21(4),494-503.
  10. Brooks, G.,Miles, J. N. V.,Torgerson, C. J.,Torgerson, D. J.(2006).Is an intervention using computer software effective in literacy learning? A randomised controlled trial.Educational Studies,32(2),133-143.
  11. Burston, J.(2015).Twenty years of MALL project implementation: A meta-analysis of learning outcomes.ReCALL,27(1),4-20.
  12. Campbell, D. T.,Fiske, D. W.(1959).Convergent and discriminant validation by the multritrait-multimethod matrix.Psychological Bulletin,56(2),81-105.
  13. Campbell, D. T.,Stanley, J. C.(1966).Experimental and quasi-experimental designs for research.Chicago, IL:Rand McNally.
  14. Chao, P.-Y.,Chen, G.-D.(2009).Augmenting paper-based learning with mobile phones.Interacting with Computers,21(3),173-185.
  15. Chen, N.-S.,Hsieh, S.-W.,Kinshuk.(2008).Effects of short-term memory and content representation type on mobile language learning.Language Learning & Technology,12(3),93-113.
  16. Chen, Y.-S.,Kao, T.-C.,Sheu, J.-P.(2005).Realizing outdoor independent learning with a butterfly-watching mobile learning system.Journal of Educational Computing Research,33(4),395-417.
  17. Cheung, A. C.-K.,Slavin, R. E.(2013).The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis.Educational Research Review,9,88-113.
  18. Chu, H.-C.,Hwang, G.-J.,Huang, S.-X.,Wu, T.-T.(2008).A knowledge engineering approach to developing e-libraries for mobile learning.Electronic Library,26(3),303-317.
  19. Cohen, J.(1994).The earth is round (p < .05).American Psychologist,49(12),997-1003.
  20. Cohen, J.(1988).Statistical power analysis for the behavioral sciences.Mahwah, NJ:Lawrence Erlbaum Associates.
  21. D'Alonzo, K. T.(2004).The Johnson-Neyman procedure as an alternative to ANCOVA.West Journal of Nursing Research,26(7),804-812.
  22. Dupont, W. D.,Plummer, W. D.(1990).Power and sample size calculations: A review and computer program.Controlled Clinical Trials,11(2),116-128.
  23. Edwards, C. M.,Rule, A. C.,Boody, R. M.(2013).Comparison of face-to-face online mathematics learning of sixth graders.Journal of Computers in Mathematics and Science Teaching,32(1),25-47.
  24. Faul, F.,Erdfelder, E.,Lang, A.-G.,Buchner, A.(2007).G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences.Behavior Research Methods,39(2),175-191.
  25. Ferguson, G. A.,Takane, Y.(1989).Statistical analysis in psychology and education.New York, NY:McGraw-Hill.
  26. Fleischer, H.(2012).What is our current understanding of one-to-one computer projects: A systematic narrative research review.Educational Research Review,7(2),107-122.
  27. Fraenkel, J. R.,Wallen, N. E.,Hyun, H.(2011).How to design and evaluate research in education.New York, NY:McGraw-Hill Education.
  28. Frohberg, D.,Goth, C.,Schwabe, G.(2009).Mobile learning projects-A critical analysis of the state of the art.Journal of Computer Assisted Learning,25(4),307-331.
  29. Gentry, T.(2008).PDAs as cognitive aids for people with multiple sclerosis.American Journal of Occupational Therapy,62(1),18-27.
  30. Glass, G. V.,Peckham, P. D.,Sanders, J. R.(1972).Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance.Review of Educational Research,42(3),237-288.
  31. Hedman, L.,Sharafi, P.(2004).Early use of internet-based educational resources: Effects on students' engagement modes and flow experience.Behaviour & Information Technology,23(2),137-146.
  32. Howell, D. C.(2002).Statistical methods for psychology.Pacific Grove, CA:Duxbury.
  33. Hsi, S.(2003).A study of user experiences mediated by nomadic web content in a museum.Journal of Computer Assisted Learning,19(3),308-319.
  34. Hsu, Y.-C.,Ching, Y.-H.(2013).Mobile computer-supported collaborative learning: A review of experimental research.British Journal of Educational Technology,44(5),E111-E114.
  35. Huck, S. W.(2008).Reading statistics and research.Boston, MA:Pearson Education.
  36. Hung, J.-L.,Zhang, K.(2012).Examining mobile learning trends 2003-2008: A categorical meta-trend analysis using text mining techniques.Journal of Computer Higher Education,24(1),1-17.
  37. Hwang, G.-J.,Shi, Y.-R.,Chu, H.-C.(2011).A concept map approach to developing collaborative mindtools for context-aware ubiquitous learning.British Journal of Educational Technology,42(5),778-789.
  38. Hwang, G.-J.,Tsai, C.-C.(2011).Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010.British Journal of Educational Technology,42(4),E65-E70.
  39. Keppel, G.(1991).Design and analysis: A researcher's handbook.Englewood Cliffs, NJ:Prentice Hall.
  40. Kerawalla, L.,O'Connor, J.,Underwood, J.,duBoulay, B.,Holmberg, J.,Luckin, R.,Tunley, H.(2007).Exploring the potential of the homework system and tablet PCs to support continuity of numeracy practices between home and primary school.Educational Media International,44(4),289-303.
  41. Kert, S. B.(2011).The use of SMS support in programming education.Turkish Online Journal of Educational Technology,10(2),268-273.
  42. Ketamo, H.(2003).An adaptive geometry game for handheld devices.Educational Technology & Society,6(1),83-95.
  43. Kirk, R. E.(1996).Practical significance: A concept whose time has come.Educational and Psychological Measurement,56(5),746-759.
  44. Kirk, R. E.(1995).Experimental design procedures for the behavioral sciences.Pacific Grove, CA:Brooks/Cole..
  45. Kondo, M.,Ishikawa, Y.,Smith, C.,Sakamoto, K.,Shimomura, H.,Wada, N.(2012).Mobile assisted language learning in university EFL courses in Japan: Developing attitudes and skills for self-regulated learning.ReCALL,24(2),169-187.
  46. Kulik, C.-L. C.,Kulik, J. A.(1991).Effectiveness of computer-based instruction: An updated analysis.Computers in Human Behavior,7(1-2),75-94.
  47. Liao, C. Y.-K.(2007).Effects of computer-assisted instruction on students' achievement in Taiwan: A meta-analysis.Computers & Education,48(2),216-233.
  48. Lin, T.-J.,Duh, H. B.-L.,Li, N.,Wang, H.-Y.,Tsai, C.-C.(2013).An investigation of learners' collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system.Computers & Education,68,314-321.
  49. Lipsey, M. W.,Wilson, D. B.(2001).Practical meta-analysis.Thousand Oaks, CA:Sage.
  50. Littell, J. H.,Corcoran, J.,Pillai, V.(2008).Systematic reviews and meta-analysis.New York, NY:Oxford University press.
  51. Liu, C.-C.,Chou, C.-C.,Liu, B.-J.,Yang, J.-W.(2006).Improving mathematics teaching and learning experiences for hard of hearing students with wireless technology-enhanced classrooms.American Annals of the Deaf,151(3),345-355.
  52. Liu, C.-C.,Hong, Y.-C.(2007).Providing hearing-impaired students with learning care after classes through smart phones and the GPRS network.British Journal of Educational Technology,38(4),727-741.
  53. Liu, T.-C.,Lin, Y.-C.,Paas, F.(2014).Effects of prior knowledge on learning from different compositions of representations in a mobile learning environment.Computers & Education,72,328-338.
  54. Liu, T.-Y.,Chu, Y.-L.(2010).Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation.Computers & Education,55(2),630-643.
  55. Looi, C.-K.,Zhang, B.,Chen, W.,Seow, P.,Chia, G.,Norris, C.,Soloway, E.(2011).1:1 mobile inquiry learning experience for primary science students: A study of learning effectiveness.Journal of Computer Assisted Learning,27(3),269-287.
  56. Mayr, S.,Erdfelder, E.,Buchner, A.,Faul, F.(2007).A short tutorial of GPower.Tutorials in Quantitative Methods for Psychology,3(2),51-59.
  57. Morris, N. P.,Ramsay, L.,Chauhan, V.(2012).Can a tablet device alter undergraduate science students' study behavior and use of technology?.Advances in Physiology Education,36(2),97-107.
  58. Oberg, A.,Daniels, P.(2013).Analysis of the effect a student-centred mobile learning instructional method has on language acquisition.Computer Assisted Language Learning,26(2),177-196.
  59. Pagano, R. R.(2007).Understanding statistics in the behavioral sciences.Belmont, CA:Thomson Higher Education.
  60. Penuel, W. R.(2006).Implementation and effects of one to one computing initiatives: A research synthesis.Journal of Research on Technology in Education,38(3),329-348.
  61. Rockinson-Szapkiw, A. J.,Courduff, J.,Carter, K.,Bennett, D.(2013).Electronic versus traditional print textbooks: A comparison study on the influence of university students' learning.Computers & Education,63,259-266.
  62. Roschelle, J.,Rafanan, K.,Bhanot, R.,Estrella, G.,Penuell, B.,Nussbaum, M.,Claro, S.(2010).Scaffolding group explanation and feedback with handheld technology: Impact on students' mathmatics learning.Educational Technology Research and Development,58(4),399-419.
  63. Sandberg, J.,Maris, M.,de Geus, K.(2011).Mobile English learning: An evidence-based study with fifth graders.Computers & Education,57(1),1334-1347.
  64. Shadish, W. R.,Cook, T. D.,Campbell, D. T.(2002).Experimental and quasi-experimental designs for generalized causal inference.Boston, MA:Houghton-Mifflin.
  65. Sink, C. A.,Mvududu, N. H.(2010).Statistical power, sampling, and effect sizes: Three keys to research relevancy.Counseling Outcome Research and Evaluation,1(2),1-18.
  66. Slavin, R. E.(2003).A reader's guide to scientifically based research.Educational Leadership,60(5),12-16.
  67. Slavin, R. E.(2002).Evidence-based education policies: Transforming educational practice and research.Educational Researcher,31(7),15-21.
  68. Slavin, R.,Madden, N. A.(2011).Measures inherent to treatments in program effectiveness reviews.Journal of Research on Educational Effectiveness,4(4),370-380.
  69. Solhaug(2009).Two configurations for accessing classroom computers: Differential impact on students' critical reflections and their empowerment.Journal of Computer Assisted Learning,25(5),411-422.
  70. Sung, Y.-T.,Chang, K.-E.,Lee, Y.-H.,Yu, W.-C.(2008).Effects of a mobile electronic guidebook on visitors' attention and visiting behaviors.Educational Technology & Society,11(2),67-80.
  71. Sung, Y.-T.,Chang, K.-E.,Liu, T.-C.(2016).The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis.Computers & Education,94,252-275.
  72. Sung, Y.-T.,Chang, K.-E.,Yang, J.-M.(2015).How effective are mobile devices for language learning? A meta-analysis.Educational Research Review,16,68-84.
  73. Tabachnick, B.,Fidell, L.(2007).Using multivariate statistics.Boston, MA:Pearson/Allyn & Bacon.
  74. Tan, T.-H.,Liu, T.-Y.,Chang, C.-C.(2007).Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning.Interactive Learning Environments,15(3),253-269.
  75. Valentine, J. C.,Cooper, H.(2008).A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The study design and implementation assessment device (Study DIAD).Psychological Methods,13(2),130-149.
  76. Valentine, J. C.,McHugh, C. M.(2007).The effects of attrition on baseline comparability in randomized experiments in education: A meta-analysis.Psychological Methods,12(3),268-282.
  77. Wolbers, K. A.,Dostal, H. M.,Graham, S.,Cihak, D.,Kilpatrick, J. R.,Saulsburry, R.(2015).The writing performance of elementary students receiving strategic and interactive writing instruction.Journal of Deaf Studies and Deaf Education,20(4),385-398.
  78. Wong, L.-H.,Looi, C.-K.(2011).What seams do we remove in mobile assisted seamless learning? A critical review of the literature.Computers & Education,57(4),2364-2381.
  79. Wu, W.-H.,Wu, Y.-C.,Chen, C.-Y.,Kao, H.-Y.,Lin, C.-H.,Huang, S.-H.(2012).Review of trends from mobile learning studies: A meta-analysis.Computers & Education,59(2),817-827.
  80. Yen, J.-C.,Lee, C.-Y.,Chen, I-J.(2012).The effects of image-based concept mapping on the learning outcomes and cognitive processes of mobile learners.British Journal of Educational Technology,43(2),307-320.
  81. Zurita, G.,Nussbaum, M.(2004).Computer supported collaborative learning using wirelessly interconnected handheld computers.Computers & Education,42(3),289-314.
  82. Zurita, G.,Nussbaum, M.,Salinas, R.(2005).Dynamic grouping in collaborative learning supported by wireless handhelds.Educational Technology & Society,8(3),149-161.
被引用次数
  1. 白景文(2020)。數位化教材導入職業訓練未來趨勢及方向。勞動及職業安全衛生研究季刊，28(2)，25-38。
  2. 黃博聖,陳姿安(2021)。微課程對注意力與學習保留之影響。教育心理學報，52(4)，885-907。
  3. 黃植懋、陳豐慈、張育愷、王俊智(2018)。身體活動與老年大腦功能：功能性磁共振造影的研究回顧。教育心理學報，50(2)，363-388。
  4. 黃芷翎(2022)。探討通識課程導入探究實作之學習成效：以仿生材料為例。通識學刊：理念與實務，10(2)，41-78。
  5. 梁桂嘉,張丹,宋曜廷(2023)。使用UEQ評估中文適性閱讀診斷網站設計之使用者體驗。商業設計學報，26，66-84。
  6. 廖遠光,陳勇欣(2021)。我國近30年科技融入數學教學對中小學生學習成效影響之後設分析。教育心理學報，52(4)，781-805。
  7. 廖遠光,楊永慈,陳政煥(2020)。行動學習對臺灣學生學業成就影響之後設分析。當代教育研究季刊，28(3)，67-102。
  8. 羅偉誠,廖信,楊翠竹,林庭卉(2023)。技術型高中設計相關類科學生應用「印刷設計課程」行動學習app之研究：以自我調整學習與學習自我效能為探究因素。教育科學研究期刊，68(2)，203-233。
  9. 羅偉誠、廖信、葉振璧(2018)。印刷專業課程應用行動學習App之教材內容設計初探。中華印刷科技年報，2018，119-129。
  10. 謝進昌,邱怡靜(2021)。個別研究品質好壞有關係嗎？類比教學對於學生科學概念改變成效後設分析。教育心理學報，52(4)，935-962。
  11. (2023)。探討擴增實境融入圖書館導覽學習系統對大學生學習成效之影響。教育傳播與科技研究，133，1-19。
print cancel