手語經驗對視覺空間心像運作影響之實驗研究

The Influence of Sign Language Experience on Imagery Abilities

10.6172/BSE.202103_46(1).0004

邱倚璿(Yi-Shiuan Chiu)

中文 ； 心像產生 ； 心像維持 ； 手語 ； 失聰／聾人 ； Chinese characters ； hearing impaired/deaf ； image generation ； image maintenance ； signed languages

特殊教育研究學刊

46卷1期（2021 / 03 / 31）

85 - 103

繁體中文

手語運用視覺空間特性呈現文法規則以達成語言溝通，本研究欲探討使用臺灣手語的使用者，相較於未有手語經驗的非手語者，是否在心像產生與心像維持運作上有所不同。本研究的心像實驗作業是以中文數字作提示，要求受試者於4×5方格中形成對應的阿拉伯數字心像，之後延宕一段時間（刺激間距ISI），於方格中的一個細格會出現一個目標項X，受試者需以按鍵判斷目標項位置是否出現在先前產生的心像路徑中。實驗操弄第一個受試者內獨變項為ISI，一組為心像產生（200、400毫秒），另一組為心像維持（800、2000毫秒）；第二個受試者內獨變項為目標項X出現在心像位置，依照筆劃順序先後分為早筆劃與晚筆劃位置，早晚筆畫差異可視為是心像完整性的指標，當有完整的心像時，早晚筆劃的差異便不復存在。實驗邀請24位失聰手語者、19位聽常手語者，與24位聽常非手語者參與，以澄清聽力、手語經驗與心像運作的關係。研究結果發現在心像產生ISI為400毫秒時，失聰手語者、聽常手語者早晚筆劃未有差異，顯示出手語使用者此時已產生完整的心像；在心像維持ISI為800毫秒時，失聰手語者與聽常手語者相較於聽常非手語者有較高的正確率，但在2000毫秒時則表現相似。基於上述結果以及失聰手語者與聽常手語者在心像運作的表現型態相似，本研究結果顯示出臺灣手語經驗與心像運作有關，手語經驗能促使手語者在心像產生與短期心像維持有其運作優勢，同時，手語和空間認知的相互作用關係可以提供更全面的觀點來了解語言如何塑造思想。

Purpose: In sign language, visual-spatial spaces are used to represent grammar rules to achieve language communication. This study examined whether the signers who had learned Taiwanese Sign Language (TSL) had different internal mental imagery processing than those who had no experience in sign languages. Methods: In this study, Chinese numerals were employed as probes in the experiment on internal mental images. The participants were required to form mental images corresponding to Arabic numbers in a 4 × 5 grid. After a period of delay (interstimulus interval [ISI]), a target X appeared in one of the grids. The participants had to determine whether the position of the target item appeared in the previously generated mental image path by pressing buttons. The first within-subject variable was ISI. One group of ISI was formed for image generation (200 and 400 ms) and the other group was formed for image maintenance (800 and 2000 ms). The second within-subject variable was the positioning of the target X in the mental image (early-imaged and late-imaged segments). The division of mental images into early-imaged and late-imaged segments was conducted according to the order of strokes in each mental image. Furthermore, on the basis of the assumption of the sequential process in image generation, the difference between early-imaged and late-imaged segments was considered an indicator of the integrity of the mental image. If the participants completed their mental images, the difference between early-imaged and late-imaged segments disappeared. The experiment recruited 24 deaf signers, 19 hearing signers, and 24 hearing nonsigners to clarify the impact of hearing loss and sign language abilities. Results/Findings: The results highlighted that during the image generation with ISI 200 ms, all participants demonstrated superior performance in the early-imaged segment condition. During the image generation with ISI 400 ms, the nonsigners still showed the effects of early-imaged and late-imaged segments. However, no differences were observed among signers in the early-imaged and late-imaged segments. This findings indicated that the signers had already produced a complete mental image by that time. When the mental image was maintained for 800 ms in the image maintenance task, the signers had a higher accuracy than the hearing nonsigners. However, the signers and nonsigners performed similarly when ISI 2000 ms was employed. The deaf signers and the hearing signers demonstrated similar performance patterns in both image tasks, highlighting the critical role played by experience in TSL. Conclusions/Implications: Our findings revealed that experience in sign language proved advantageous for signers in terms of mental image generation and short-term mental image maintenance. The interaction of sign languages and spatial cognition could offer a more comprehensive view of how language shapes the mind.

1. 劉秀丹, Hsiu-Tan,劉俊榮, Chun-Jung,曾進興, Chin-Hsing,張榮興, Tung-Hsing(2015)。臺灣手語理解能力標準化測驗的編製與發展。特殊教育研究學刊，40(3)，27-57。
   連結：
2. Albers, A. M.,Kok, P.,Toni, I.,Dijkerman, H. C.,de Lange, F. P.(2013).Shared representations for working memory and mental imagery in early visual cortex.Current Biology,23(15),1427-1431.
3. Bellugi, U.,O’Grady, L.,Lillo-Martin, D.,O’Grady Hynes, M.,van Hoek, K.,Corina, D.(1990).Enhancement of spatial cognition in deaf children.From gesture to language in hearing and deaf children,Berlin/Heidelberg, Germany:
4. Bialystok, E.(2009).Bilingualism: The good, the bad, and the indifferent.Bilingualism: Language and Cognition,12(1),3-11.
5. Bialystok, E.(2017).The bilingual adaptation: How minds accommodate experience.Psychological Bulletin,143(3),233-262.
6. Borst, G.,Ganis, G.,Thompson, W. L.,Kosslyn, S. M.(2012).Representations in mental imagery and working memory: Evidence from different types of visual masks.Memory and Cognition,40(2),204-217.
7. Borst, G.,Kosslyn, S. M.(2008).Visual mental imagery and visual perception: Structural equivalence revealed by scanning processes.Memory and Cognition,36(4),849-862.
8. Chamberlain, C.,Mayberry, R. I.(1994).Do the deaf "see'' better? Effects of deafness on visuospatial skills.TENNET V meetings,Montreal, Canada:
9. Conlin, D.,Paivio, A.(1975).The associative learning of the deaf: The effects of word imagery and signability.Memory and Cognition,3(3),335-340.
10. De Beni, R.,Pazzaglia, F.,Gardini, S.(2007).The generation and maintenance of visual mental images: Evidence from image type and aging.Brain and cognition,63(3),271-278.
11. Denis, M.,Kosslyn, S. M.(1999).Scanning visual mental images: A window on the mind.Cahiers de Psychologie Cognitive,18(4),409-465.
12. Emmorey, K.,Klima, E.,Hickok, G.(1998).Mental rotation within linguistic and nonlinguistic domains in users of American sign language.Cognition,68(3),221-246.
13. Emmorey, K.,Kosslyn, S. M.(1996).Enhanced image generation abilities in deaf signers: A right hemisphere effect.Brain and Cognition,32(1),28-44.
14. Emmorey, K.,Kosslyn, S. M.,Bellugi, U.(1993).Visual imagery and visual-spatial language: Enhanced imagery abilities in deaf and hearing ASL signers.Cognition,46(2),139-181.
15. Emmorey, K.,Luk, G.,Pyers, J. E.,Bialystok, E.(2008).The source of enhanced cognitive control in bilinguals: Evidence from bimodal bilinguals.Psychological Science,19(12),1201-1206.
16. Emmorey, K.,Secora, K.,Brozdowski, C.(2019).Assessing the comprehension of spatial perspectives in ASL classifier constructions.The Journal of Deaf Studies and Deaf Education,24(3),214-222.
17. Flaherty, M.(2003).Sign language and Chinese characters on visual-spatial memory: A literature review.Perceptual and Motor Skills,97(3),797-802.
18. Flaherty, M.(2000).Memory in the deaf: A crosscultural study in English and Japanese.American Annals of the Deaf,145(3),237-244.
19. Ganis, G.,Thompson, W. L.,Kosslyn, S. M.(2004).Brain areas underlying visual mental imagery and visual perception: an fMRI study.Cognitive Brain Research,20(2),226-241.
20. Hirsch, C. R.,Clark, D. M.,Mathews, A.,Williams, R.(2003).Self-images play a causal role in social phobia.Behaviour Research and Therapy,41(8),909-921.
21. Holmes, E. A.,Arntz, A.,Smucker, M. R.(2007).Imagery rescripting in cognitive behaviour therapy: Images, treatment techniques and outcomes.Journal of Behavior Therapy and Experimental Psychiatry,38(4),297-305.
22. Joffe, V. L.,Cain, K.,Marić, N.(2007).Comprehension problems in children with specific language impairment: does mental imagery training help?.International Journal of Language and Communication Disorders,42(6),648-664.
23. Keogh, R.,Pearson, J.(2014).The sensory strength of voluntary visual imagery predicts visual working memory capacity.Journal of Vision,14(12),1-13.
24. Kosslyn, S. M.(1994).Image and brain.Cambridge, MA:MIT Press.
25. Kosslyn, S. M.,Cave, C. B.,Provost, D. A.,von Gierke, S. M.(1988).Sequential processes in image generation.Cognitive Psychology,20(3),319-343.
26. Kosslyn, S. M.,Ganis, G.,Thompson, W. L.(2001).Neural foundations of imagery.Nature Reviews Neuroscience,2(9),635-642.
27. Kosslyn, S. M.,Margolis, J. A.,Barrett, A. M.,Goldknopf, E. J.,Daly, P. F.(1990).Age differences in imagery abilities.Child Development,61(4),995-1010.
28. Kosslyn, S. M.,Thompson, W. L.,Ganis, G.(2006).The case for mental imagery.Oxford, UK:Oxford University Press.
29. Kozhevnikov, M.,Motes, M. A.,Hegarty, M.(2007).Spatial visualization in physics problem solving.Cognitive Science,31(4),549-579.
30. Laeng, B.,Sulutvedt, U.(2013).The eye pupil adjusts to imaginary light.Psychological Science,25(1),188-197.
31. Lai, M. Y.,Leung, F. K. S.(2012).Visual perceptual abilities of Chinese-speaking and English-speaking children.Perceptual and Motor Skills,114(2),433-445.
32. Nippold, M. A.,Duthie, J. K.(2003).Mental imagery and idiom comprehension.Journal of Speech, Language, and Hearing Research,46(4),788-799.
33. O'Craven, K. M.,Kanwisher, N.(2000).Mental imagery of faces and places activates corresponding stiimulus-specific brain regions.Journal of Cognitive Neuroscience,12(6),1013-1023.
34. Ormrod, J. E.(1985).Visual memory in a spelling matching task: Comparison of good and poor spellers.Perceptual and Motor Skills,61(1),183-188.
35. Palermo, L.,Piccardi, L.,Nori, R.,Giusberti, F.,Guariglia, C.(2016).The impact of ageing and gender on visual mental imagery processes: A study of performance on tasks from the Complete Visual Mental Imagery Battery (CVMIB).Journal of Clinical and Experimental Neuropsychology,38(7),752-763.
36. Pearson, J.,Kosslyn, S. M.(2015).The heterogeneity of mental representation: Ending the imagery debate.Proceedings of the National Academy of Sciences of the United States of America,112(33),10089-10092.
37. Pearson, J.,Naselaris, T.,Holmes, E. A.,Kosslyn, S. M.(2015).Mental imagery: Functional mechanisms and clinical applications.Trends in cognitive sciences,19(10),590-602.
38. Piccardi, L.,Nori, R.,Palermo, L.,Guariglia, C.,Giusberti, F.(2015).Age effect in generating mental images of buildings but not common objects.Neuroscience Letters,602,79-83.
39. Podgorny, P.,Shepard, R. N.(1978).Functional representations common to visual perception and imagination.Journal of Experimental Psychology: Human Perception and Performance,4(1),21-35.
40. Rademaker, R.,Pearson, J.(2012).Training visual imagery: Improvements of metacognition, but not imagery strength.Frontiers in Psychology,3(224),1-11.
41. Shen, H. H.(2010).Imagery and verbal coding approaches in Chinese vocabulary instruction.Language Teaching Research,14(4),485-499.
42. Shepard, R. N.,Metzler, J.(1971).Mental rotation of three-dimensional objects.Science,171(3972),701-703.
43. Stevenson, H. W.,Stigler, J. W.,Lee, S. Y.,Lucker, G. W.,Kitamura, S.,Hsu, C. C.(1985).Cognitive Performance and Academic Achievement of Japanese, Chinese, and American Children.Child Development,56(3),718-734.
44. Sugishita, M.,Omura, K.(2001).Learning Chinese characters may improve visual recall.Perceptual and Motor Skills,93(3),579-594.
45. Talbot, K. F.,Haude, R. H.(1993).The relation between sign language skill and spatial visualization ability: Mental rotation of three dimensional objects.Perceptual and Motor Skills,77(3_suppl),1387-1391.
46. Tartaglia, E. M.,Bamert, L.,Mast, F. W.,Herzog, M. H.(2009).Human perceptual learning by mental imagery.Current Biology,19(24),2081-2085.
47. Uttal, D. H.,Cohen, C. A.(2012).Spatial thinking and STEM education: When, why, and how?.Psychology of Learning and Motivation,Cambridge, MA:
48. Wai, J.,Lubinski, D.,Benbow, C. P.(2009).Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance.Journal of educational Psychology,101(4),817.
49. Williams, A. D.,Blackwell, S. E.,Mackenzie, A.,Holmes, E. A.,Andrews, G.(2013).Combining imagination and reason in the treatment of depression: A randomized controlled trial of internet-based cognitivebias modification and internet-CBT for depression.Journal of Consulting and Clinical Psychology,81(5),793-799.
50. Zeman, A. Z. J.,Della Sala, S.,Torrens, L. A.,Gountouna, V. -E.,McGonigle, D. J.,Logie, R. H.(2010).Loss of imagery phenomenology with intact visuo-spatial task performance: A case of ‘blind imagination’.Neuropsychologia,48(1),145-155.