英文摘要
|
Text and diagrams are frequently presented together in science textbooks and popular science articles. Text is used to describe concepts, and diagrams contain visual and spatial information depicting conceptual relationships, object structures, and developmental processes. Two primary theoretical models have been widely adopted by researchers in the domain of text-diagram comprehension: Mayer's cognitive theory of multimedia learning (CTML; Mayer, 2005, 2014) and Schnotz's integrated model of text and picture comprehension (ITPC; Schnotz & Bannert, 2003; Schnotz et al., 2014). In contrast to findings of benefits associated with Mayer's "multimedia principle" (Mayer, 2005, 2014), several studies have documented neutral or negative learning effects from studying images (McTigue, 2009; Segers et al., 2008). Previous literature reviews on text-diagram reading have either been published more than 10 years ago (Carney & Levin, 2002; Phillips et al., 2010; Vekiri, 2002) or mostly focused on offline outcome measures (Carney & Levin, 2002; Guo et al., 2020; Phillips et al., 2010; Schnotz, 2014), which may not reflect the complexity of text-diagram processing. Empirical evidence of eye movements during text-diagram reading has been obtained in several studies, but no systematic review has been conducted to synthesize these findings. This systematic review was designed to synthesize the empirical research findings of eye-tracking studies in the domain of text-diagram science reading over the past 30 years. The three specific research questions were the following: a. What are the readers' eye movement patterns, and what is the relationship between these patterns and learning performance in text-diagram science reading? b. How and under what conditions do potential influencing factors affect readers' eye movement patterns and performance in text-diagram science reading? c. Can interventions affect eye movement patterns and performance in text-diagram science reading? A systematic literature review was conducted using a methodological three-step process (Guo et al., 2020). a. Identifying and searching articles published between January 1990 and May 2020 in Scopus, Education Database (ProQuest), ERIC (ProQuest), PsycArticle, and Airiti Library (Chinese) by using a combination of multiple key terms: eye-tracking (eye-tracking OR eye-movement OR eye-fixation), diagram (figure* OR illustration* OR graph* OR picture* OR diagram*), and reading (reading OR text* OR multimedia). After duplicates were excluded, 579 English-language articles and 33 Chinese-language articles were identified during the initial search. b. Screening and coding studies using predetermined selection criteria: (a) examined empirically; (b) published in a peer-reviewed SCIE, SSCI, TSSCI, or THCI journal; (c) measured participants' eye movements and learning outcomes; (d) involved text-diagram reading in the science domain; (e) primarily focused on reading comprehension (studies involving problem-solving or reasoning tasks were excluded); (f) used static visual displays as materials (studies using video, audio, simulation, computer games, and interactive diagrams were excluded). After abstract and full-text screening, 50 English-language articles (including 54 studies) and one Chinese-language article were retained for inclusion in the analysis. c. Analyzing the included studies and interpreting findings using inductive paradigmatic analysis (Polkinghorne, 1995). Based on Guo et al. (2020), categories were not predetermined by the researchers but rather were identified in an inductive manner. After descriptive information was extracted from the included studies and tentatively coded, we compared the similarities and differences of the codes and categorized them into themes to answer the research questions. Finally, the research findings were presented on the basis of these different themes. Of the 55 studies, 32 were conducted with college students. Regarding content area, more than half of the studies involved biology and medicine (n = 34), and mechanics and physics (n = 13) and earth sciences and geography (n = 10) represented the second and third most common content areas. Articles were most commonly sourced from the journals Learning and Instruction (n = 6) and Computers & Education (n = 5). SMI RED 250 (n = 10), EyeLink 1000 (n = 10), and Tobii T120 (n = 9) were the most common models of eye-trackers used. The most commonly used eye movement indicators were total fixation duration (n = 41) and the transition between areas of interest (n = 36). The qualitative synthesis of the studies was organized according to two major themes: cognitive processes and interventions. a. Cognitive processes. (1) Eye movement patterns. The eye movement data support dual-coding theory (Paivio, 1986), which is the theoretical basis of CTML and ITPC and suggests that the human cognitive system is composed of two sets of subsystems: verbal and pictorial. Generally, reading is text-driven rather than diagram-driven, and readers may have different reading patterns that involve varying degrees of diagram processing and text-diagram integration. Those who more often inspect diagrams and refer to text and diagrams during reading usually performed better. (2) Reader characteristics. Higher-level readers were more likely to utilize diagram inspection and text-diagram referencing tactics compared with lower-level readers, and they generally had better learning performance; this applied to students in both academic-oriented schools (compared to non-academic-oriented schools) and high-ability students (compared to low-ability students). People with low prior knowledge (PK) tended to inspect more text than those with high PK, but no consistent pattern was discernible in diagram inspections, text-diagram referencing, and learning performance. In terms of cognitive style, visualizers fixated more on diagrams, and verbalizers fixated more on text, but the findings regarding learning performance were mixed. (3) Material characteristics and their interactions with reader characteristics. High-ability readers put more effort into difficult articles, and low-ability readers put more effort into articles with medium-low difficulty. High-ability and senior-grade readers increased diagram inspection and "diagram-item" references as the difficulty of the task increased. Readers in the second cycle of reading decreased text inspection and increased diagram inspection, reflecting adjustment by the reader. When inconsistent information appeared in material, readers typically performed more inspections, reinspections, and transitions in the beginning, but this did not affect reading performance. b. Interventions. (1) Material manipulation. First, readers tended to fixate on signaled sections (i.e., key points were highlighted via colors, labels, and arrows) faster than on nonsignaled sections, and this effect was greater for readers with low PK. Readers performed better on signaled materials than on nonsignaled materials, but this pattern did not show among younger readers. Second, readers inspected explanatory diagrams more than decorative diagrams; however, the effect on performance by adding different types of diagrams was unclear. Third, readers transitioned more between text and diagrams when the two were physically integrated than when they were separated. The spatial contiguity effect on performance depended on whether the information was necessary. (2) Instructional interventions. Overall, instructional intervention usually affected readers' eye movement patterns and was likely to enhance learning performance through mediators of visual behavior such as text-diagram references and diagram inspection. Reading ability and domain knowledge may be moderating variables. Our results demonstrated that the inclusion of diagrams in science texts usually affects readers' visual behavior but does not guarantee positive learning performance. The key to learning improvement is the effective operation of the dual channels of a reader's cognitive system. Some reader characteristics (such as ability and PK) may play moderating roles in eye movement and learning performance. Several gaps in eye-tracking research on text-diagram science reading were identified in this review. First, more research studies should be conducted with preschool children, K-12 students, and adults. Second, future research studies should include spatial-scale eye movement indicators and distinguish between initial and late processing indicators. Third, other process measures, such as think-aloud protocols, should be included to help interpret eye movement data. Fourth, future research should focus on potential moderators (such as ability, PK, age, and content area) or the factors that contain complex research manipulations and were underrepresented in the studies in our review (such as cognitive style and type of diagram). Additionally, it is recommended that future research further explore readers' self-regulatory processes by using eye-tracking technology. Finally, the findings of this study suggest that instructors should also consider individual differences and modify their instructions to help readers master the skills of reading diagrams and text-diagram references rather than focusing exclusively on the course content. First, the strict screening criteria used in this study may have limited the scope of this review. Second, some empirical studies were based on small samples or a single measurement, which may have limited the findings, and the data pertaining to some themes in the review came from a small number of studies; therefore, special care must be taken when interpreting the results. Third, the empirical research itself may have been somewhat biased when published, which may have inflated the effectiveness of the interventions. Fourth, this review focused on the domain of scientific reading materials, and consequently, the results may not be generalized to other content areas.
|
参考文献
|
-
呂昕頤, H.-I.,詹雨臻, Y.-C.,陳學志, H.-C.(2019)。笑話中歧義與推論歷程之眼動分析。教育心理學報,50,587-609。
連結:
-
官英華, Y.-H.(2020)。語音轉碼對臺灣大學生閱讀歷程的影響:來自眼動實驗的證據。教育心理學報,52,459-488。
連結:
-
陳學志, H.-C.,賴惠德, H.-D.,邱發忠, F.-C.(2010)。眼球追蹤技術在學習與教育上的應用。教育科學研究期刊,55(4),39-68。
連結:
-
簡郁芩, Y.-C.,吳昭容, C.-J.(2012)。以眼動型態和閱讀測驗表現探討箭頭在科學圖文閱讀中的圖示效果。中華心理學刊,54,385-402。
連結:
-
Alemdag, E.,Cagiltay, K.(2018).A systematic review of eye tracking research on multimedia learning.Computers & Education,125,413-428.
-
Bol, N.,van Weert, J. C. M.,Loos, E. F.,Romano Bergstrom, J. C.,Bolle, S.,Smets, E. M. A.(2016).How are online health messages processed? Using eye tracking to predict recall of information in younger and older adults.Journal of Health Communication,21,387-396.
-
Butcher, K. R.(2014).The multimedia principle.The Cambridge handbook of multimedia learning
-
Carney, R. N.,Levin, J. R.(2002).Pictorial illustrations still improve students’ learning from text.Educational Psychology Review,14,5-26.
-
Chuang, H.-H.,Liu, H.-C.(2012).Effects of different multimedia presentations on viewers’ information-processing activities measured by eye-tracking technology.Journal of Science Education and Technology,21,276-286.
-
Coleman, J. M.,McTigue, E. M.,Dantzler, J. A.(2018).What makes a diagram easy or hard? The impact of diagram design on fourth-grade students’ comprehension of science texts.The Elementary School Journal,119,122-151.
-
Cromley, J. G.,Snyder-Hogan, L. E.,Luciw-Dubas, U. A.(2010).Cognitive activities in complex science text and diagrams.Contemporary Educational Psychology,35,59-74.
-
Eitel, A.(2016).How repeated studying and testing affects multimedia learning: Evidence for adaptation to task demands.Learning and Instruction,41,70-84.
-
Guo, D.,McTigue, E. M.,Matthews, S. D.,Zimmer, W.(2020).The impact of visual displays on learning across the disciplines: A systematic review.Educational Psychology Review,32,627-656.
-
Guo, D.,Zhang, S.,Wright, K. L.,McTigue, E. M.(2020).Do you get the picture? A meta-analysis of the effect of graphics on reading comprehension.AERA Open,6(1)
-
Hannus, M.,Hyönä, J.(1999).Utilization of illustrations during learning of science textbook passages among low- and high-ability children.Contemporary Educational Psychology,24,95-123.
-
Harp, S. F.,Mayer, R. E.(1998).How seductive details do their damage: A theory of cognitive interest in science learning.Journal of Educational Psychology,90,414-434.
-
Hegarty, M.,Just, M. A.(1993).Constructing mental models of machines from text and diagrams.Journal of Memory and Language,32,717-742.
-
Ho, H. N. J.,Tsai, M.-J.,Wang, C.-Y.,Tsai, C.-C.(2014).Prior knowledge and online inquiry-based science reading: Evidence from eye tracking.International Journal of Science and Mathematics Education,12,525-554.
-
Hochpöchler, U.,Schnotz, W.,Rasch, T.,Ullrich, M.,Horz, H.,McElvany, N.,Baumert, J.(2013).Dynamics of mental model construction from text and graphics.European Journal of Psychology of Education,28,1105-1126.
-
Höffler, T. N.(2010).Spatial ability: Its influence on learning with visualizations—a meta-analytic review.Educational Psychology Review,22,245-269.
-
Höffler, T. N.,Koć-Januchta, M.,Leutner, D.(2017).More evidence for three types of cognitive style: Validating the object-spatial imagery and verbal questionnaire using eye tracking when learning with texts and pictures.Applied Cognitive Psychology,31,109-115.
-
Holmqvist, K.,Nyström, M.,Andersson, R.,Dewhurst, R.,Jarodzka, H.,Van de Weijer, J.(2011).Eye tracking: A comprehensive guide to methods and measures.Oxford University Press.
-
Hung, Y.-N.(2014).“What are you looking at?” An eye movement exploration in science text reading.International Journal of Science and Mathematics Education,12,241-260.
-
Hyönä, J.,Lorch, R. F., Jr.,Rinck, M.(2003).Eye movement measures to study global text processing.The mind’s eye: Cognitive and applied aspects of eye movement research
-
Jian, Y.-C.(2017).Eye-movement patterns and reader characteristics of students with good and poor performance when reading scientific text with diagrams.Reading and Writing,30,1447-1472.
-
Jian, Y.-C.(2016).Fourth graders’ cognitive processes and learning strategies for reading illustrated biology texts: Eye movement measurements.Reading Research Quarterly,51,93-109.
-
Jian, Y.-C.(2018).Reading instructions influence cognitive processes of illustrated text reading not subject perception: An eye-tracking study.Frontiers in Psychology,9,Article 2263.
-
Jian, Y.-C.(2019).Reading instructions facilitate signaling effect on science text for young readers: An eye-movement study.International Journal of Science and Mathematics Education,17,503-522.
-
Jian, Y.-C.,Chen, M.-L.,Ko, H.-W.(2013).Context effects in processing of Chinese academic words: An eye-tracking investigation.Reading Research Quarterly,48,403-413.
-
Jian, Y.-C.,Ko, H.-W.(2017).Influences of text difficulty and reading ability on learning illustrated science texts for children: An eye movement study.Computers & Education,113,263-279.
-
Jian, Y.-C.,Su, J.-H.,Hsiao, Y.-R.(2019).Differentiated processing strategies for science reading among sixth-grade students: Exploration of eye movements using cluster analysis.Computers & Education,142,Article 103652.
-
Jian, Y.-C.,Wu, C.-J.(2015).Using eye tracking to investigate semantic and spatial representations of scientific diagrams during text-diagram integration.Journal of Science Education and Technology,24,43-55.
-
Jian, Y.-C.,Wu, C.-J.(2016).The function of diagram with numbered arrows and text in helping readers construct kinematic representations: Evidenced from eye movements and reading tests.Computers in Human Behavior,61,622-632.
-
Johnson, C. I.,Mayer, R. E.(2012).An eye movement analysis of the spatial contiguity effect in multimedia learning.Journal of Experimental Psychology: Applied,18,178-191.
-
Just, M. A.,Carpenter, P. A.(1980).A theory of reading: From eye fixations to comprehension.Psychological Review,87,329-354.
-
Kalyuga, S.,Ayres, P.,Chandler, P.,Sweller, J.(2003).The expertise reversal effect.Educational Psychologist,38,23-31.
-
Ketzer-Nöltge, A.,Schweppe, J.,Rummer, R.(2019).Is the seductive details effect moderated by mood? An eye-tracking study.Applied Cognitive Psychology,33,62-70.
-
Koć-Januchta, M.,Höffler, T.,Thoma, G.-B.,Prechtl, H.,Leutner, D.(2017).Visualizers versus verbalizers: Effects of cognitive style on learning with texts and pictures – An eye-tracking study.Computers in Human Behavior,68,170-179.
-
Korbach, A.,Ginns, P.,Brünken, R.,Park, B.(2020).Should learners use their hands for learning? Results from an eye-tracking study.Journal of Computer Assisted Learning,36,102-113.
-
Krebs, M. C.,Schüler, A.,Scheiter, K.(2019).Just follow my eyes: The influence of model-observer similarity on Eye Movement Modeling Examples.Learning and Instruction,61,126-137.
-
Lai, M.-L.,Tsai, M.-J.,Yang, F.-Y.,Hsu, C.-Y.,Liu, T.-C.,Lee, S. W.-Y.,Lee, M.-H.,Chiou, G.-L.,Liang, J.-C.,Tsai, C.-C.(2013).A review of using eye-tracking technology in exploring learning from 2000 to 2012.Educational Research Review,10,90-115.
-
Lin, Y. Y.,Holmqvist, K.,Miyoshi, K.,Ashida, H.(2017).Effects of detailed illustrations on science learning: An eye-tracking study.Instructional Science,45,557-581.
-
Liu, C.-J.,Kemper, S.,McDowd, J.(2009).The use of illustration to improve older adults’ comprehension of health-related information: Is it helpful?.Patient Education and Counseling,76,283-288.
-
Liu, H.-C.(2018).Investigating the impact of cognitive style on multimedia learners’ understanding and visual search patterns: An eye-tracking approach.Journal of Educational Computing Research,55,1053-1068.
-
Liversedge, S. P.,Findlay, J. M.(2000).Saccadic eye movements and cognition.Trends in Cognitive Sciences,4,6-14.
-
Mason, L.,Pluchino, P.,Tornatora, M. C.(2016).Using eye-tracking technology as an indirect instruction tool to improve text and picture processing and learning.British Journal of Educational Technology,47,1083-1095.
-
Mason, L.,Pluchino, P.,Tornatora, M. C.(2013).Effects of picture labeling on science text processing and learning: Evidence from eye movements.Reading Research Quarterly,48,199-214.
-
Mason, L.,Pluchino, P.,Tornatora, M. C.(2015).Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning.Contemporary Educational Psychology,41,172-187.
-
Mason, L.,Pluchino, P.,Tornatora, M. C.,Ariasi, N.(2013).An eye-tracking study of learning from science text with concrete and abstract illustrations.The Journal of Experimental Education,81,356-384.
-
Mason, L.,Scheiter, K.,Tornatora, M. C.(2017).Using eye movements to model the sequence of text–picture processing for multimedia comprehension.Journal of Computer Assisted Learning,33,443-460.
-
Mason, L.,Tornatora, M. C.,Pluchino, P.(2015).Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: An eye-movement study.Reading and Writing,28,851-872.
-
Mason, L.,Tornatora, M. C.,Pluchino, P.(2013).Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns.Computers & Education,60,95-109.
-
Mayer, R. E.(2005).Cognitive theory of multimedia learning.The Cambridge handbook of multimedia learning
-
Mayer, R. E.(2014).The Cambridge handbook of multimedia learning.Cambridge University Press.
-
Mayer, R. E.,Gallini, J. K.(1990).When is an illustration worth ten thousand words?.Journal of Educational Psychology,82,715-726.
-
McTigue, E. M.(2009).Does multimedia learning theory extend to middle-school students?.Contemporary Educational Psychology,34,143-153.
-
Meppelink, C. S.,Bol, N.(2015).Exploring the role of health literacy on attention to and recall of text-illustrated health information: An eye-tracking study.Computers in Human Behavior,48,87-93.
-
Morrow, D.,D’andrea, L.,Stine-Morrow, E. A. L.,Shake, M.,Bertel, S.,Chin, J.,Kopren, K.,Gao, X.,Conner-Garcia, T.,Graumlich, J.,Murray, M.(2012).Comprehension of multimedia health information among older adults with chronic illness.Visual Communication,11,347-362.
-
Mudrick, N. V.,Azevedo, R.,Taub, M.(2019).Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning.Computers in Human Behavior,96,223-234.
-
Norman, R. R.(2012).Reading the graphics: What is the relationship between graphical reading processes and student comprehension?.Reading and Writing,25,739-774.
-
Ozcelik, E.,Karakus, T.,Kursun, E.,Cagiltay, K.(2009).An eye-tracking study of how color coding affects multimedia learning.Computers & Education,53,445-453.
-
Paivio, A.(1986).Mental representations: A dual coding approach.Oxford University Press.
-
Parkhurst, P. E.,Dwyer, F. M.(1983).An experimental assessment of students’ IQ level and their ability to profit from visualized instruction.Journal of Instructional Psychology,10,9-20.
-
Phillips, L. M.,Norris, S. P.,Macnab, J. S.(2010).The concept of visualization.Visualization in mathematics, reading and science education
-
Polkinghorne, D. E.(1995).Narrative configuration in qualitative analysis.International Journal of Qualitative Studies in Education,8,5-23.
-
Rayner, K.(1998).Eye movements in reading and information processing: 20 years of research.Psychological Bulletin,124,372-422.
-
Renkl, A.,Scheiter, K.(2017).Studying visual displays: How to instructionally support learning.Educational Psychology Review,29,599-621.
-
Richter, J.,Scheiter, K.(2019).Studying the expertise reversal of the multimedia signaling effect at a process level: Evidence from eye tracking.Instructional Science,47,627-658.
-
Rop, G.,Schüler, A.,Verkoeijen, P. P. J. L.,Scheiter, K.,Van Gog, T.(2018).Effects of task experience and layout on learning from text and pictures with or without unnecessary picture descriptions.Journal of Computer Assisted Learning,34,458-470.
-
Rop, G.,Schüler, A.,Verkoeijen, P. P. J. L.,Scheiter, K.,Van Gog, T.(2018).The effect of layout and pacing on learning from diagrams with unnecessary text.Applied Cognitive Psychology,32,610-621.
-
Sanchez, C. A.,Wiley, J.(2006).An examination of the seductive details effect in terms of working memory capacity.Memory & Cognition,34,344-355.
-
Scheiter, K.,Eitel, A.(2015).Signals foster multimedia learning by supporting integration of highlighted text and diagram elements.Learning and Instruction,36,11-26.
-
Scheiter, K.,Schubert, C.,Schüler, A.(2018).Self‐regulated learning from illustrated text: Eye movement modelling to support use and regulation of cognitive processes during learning from multimedia.British Journal of Educational Psychology,88,80-94.
-
Scheiter, K.,Schubert, C.,Schüler, A.,Schmidt, H.,Zimmermann, G.,Wassermann, B.,Krebs, M. C.,Eder, T.(2019).Adaptive multimedia: Using gaze-contingent instructional guidance to provide personalized processing support.Computers & Education,139,31-47.
-
Schnotz, W.(2014).Integrated model of text and picture comprehension.The Cambridge handbook of multimedia learning
-
Schnotz, W.,Bannert, M.(2003).Construction and interference in learning from multiple representation.Learning and Instruction,13,141-156.
-
Schnotz, W.,Ludewig, U.,Ullrich, M.,Horz, H.,McElvany, N.,Baumert, J.(2014).Strategy shifts during learning from texts and pictures.Journal of Educational Psychology,106,974-989.
-
Schnotz, W.,Wagner, I.(2018).Construction and elaboration of mental models through strategic conjoint processing of text and pictures.Journal of Educational Psychology,110,850-863.
-
Schüler, A.(2019).The integration of information in a digital, multi-modal learning environment.Learning and Instruction,59,76-87.
-
Schüler, A.(2017).Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration.Learning and Instruction,49,218-231.
-
Segers, E.,Verhoeven, L.,Hulstijn-Hendrikse, N.(2008).Cognitive processes in children’s multimedia text learning.Applied Cognitive Psychology,22,375-387.
-
Stalbovs, K.,Scheiter, K.,Gerjets, P.(2015).Implementation intentions during multimedia learning: Using if-then plans to facilitate cognitive processing.Learning and Instruction,35,1-15.
-
Stárková, T.,Lukavský, J.,Javora, O.,Brom, C.(2019).Anthropomorphisms in multimedia learning: Attract attention but do not enhance learning?.Journal of Computer Assisted Learning,35,555-568.
-
Vekiri, I.(2002).What is the value of graphical displays in learning?.Educational Psychology Review,14,261-312.
-
Zander, S.,Wetzel, S.,Kühl, T.,Bertel, S.(2017).Underlying processes of an inverted personalization effect in multimedia learning – An eye-tracking study.Frontiers in Psychology,8,Article 2202.
-
Zhao, F.,Schnotz, W.,Wagner, I.,Gaschler, R.(2020).Texts and pictures serve different functions in conjoint mental model construction and adaptation.Memory & Cognition,48,69-82.
|