透過台灣手語像似性來瞭解手部動作的意義

Understanding the Meaning of the Hand Movements through the Iconicity of Taiwan Sign Language

10.6129/CJP.20121014a

邱倚璿(Yi-Shiuan Chiu)；郭文瑞(Wen-Jui Kuo)；曾志朗(Ovid J. L. Tzeng)

手勢 ； 台灣手語 ； 像似性 ； 語意 ； 體現認知 ； embodied cognition ； gesture ； iconicity ； semantic ； Taiwan Sign Language

中華心理學刊

55卷1期（2013 / 03 / 01）

119 - 138

繁體中文

本研究主要在探討體現經驗與語言知識，如何對像似性手語動作中的理解與產生造成影響，藉此瞭解人們是如何理解手語動作之意義。在實驗一，手語生手對於手語專家分類的高低像似性手語作像似性程度的評分，結果顯示兩者之間有顯著的共通性；實驗二中，操弄手語像似性程度（高、低），要求手語生手對每一個手語之五種可能意義作排序，並對各選項排序作信心評分，結果發現專家評為高像似性的手語，確實較易被手語生手選出正確意義，而理解低像似性手語時，會傾向運用語意類別來幫助意義的猜測；實驗三中，除了操弄像似性程度與五種可能意義外，再加上像似性表徵特性（操作為主、視覺為主），結果發現在高像似性手語中，操作表徵特性與視覺表徵特性對於正確意義的猜測同等重要，但操作為主的手語較能容許不同語意類別的可能意義；實驗四中，手語生手須對具體名詞圖片產生一個對應的手勢動作，並記錄圖片開始呈現至手指釋放按鍵以產生手勢動作的反應時間，結果顯示手語生手對於操作為主且高像似性圖片可較快產生合適的手勢。本研究結果顯示體現經驗對動作理解扮演著重要的角色，而語言知識的角色較不重要。

We investigated the influences of embodied experiences and language-dependent knowledge on comprehending the iconic signs of Taiwan Sign Language (TSL), and the method of judging the possible meanings of iconic signs by naïve signers. In Experiment 1, we manipulated the degree of iconicity categorized by TSL experts and asked naïve signers to do a subjective iconicity rating task. The results showed similar judgments of iconicity degree between experts and naïve signers. In Experiment 2, iconicity was investigated by a 5-possible-meaning ranking task and a confidence rating task for each meaning. These results revealed that the meanings of the high iconic (HI) signs are easier to correctly determine, and the ranking performance of low iconic (LI) signs is influenced by the semantic categories. In Experiment 3, we accessed the representations of iconicity (manipulation-based/M, vision-based/V). While correctly ranking HI meaning, naïve signers can tolerate meanings from different semantic categories when viewing the M signs. In Experiment 4, naïve signers were asked to do picture gesturing of concrete nouns and the time from the onset of the picture to the release of the button was analyzed. The data suggested that manipulation-based and HI pictures are faster to gesture than others. Drawn from these observations, we demonstrated that the degree and the representations of iconicity in TSL can cross culture boundaries. Our bodily experience plays a prominent role in comprehending gestures.

1. Word list with accumulated word frequency in Sinica corpus 3.0. (1998). Taipei, Taiwan: The Association for Computational Linguistics and Chinese Language Processing..
2. Armstrong, D. F.(2008).The gestural theory of language origins.Sign Language Studies,8,289-314.
3. Bates, E.,D''Amico, S.,Jacobsen, T.,Székely, A.,Andonova, E.,Devescovi, A.(2003).Timed picture naming in seven languages.Psychonomic Bulletin and Review,10,344-380.
4. Beattie, G.,Coughlan, J.(1999).An experimental investigation of the role of iconic gestures in lexical access using the tip-of-the-tongue phenomenon.British Journal of Psychology,90,35-56.
5. Bosworth, R. G.,Emmorey, K.(2010).Effects of iconicity and semantic relatedness on lexical access in American Sign Language.Journal of Experimental Psychology: Learning, Memory and Cognition,36,1573-1581.
6. Buccino, G.,Binkofski, F.,Fink, G. R.,Fadiga, L.,Fogassi, L.,Gallese, V.(2001).Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study.European Journal of Neuroscience,13,400-404.
7. Campbell, R.(Ed.),Messing, L. S.(Ed.)(1999).Gesture, speech, and sign.Oxford, UK:Oxford University Press.
8. Chiu, Y.-H.,Hsieh, J.-C.,Kuo, W.-J.,Hung, D. L.,Tzeng, O. J.-L.(2005).A study of the vision- and manipulation-based signs in Taiwan sign language.Language and Linguistics,6,361-380.
9. Diessel, H.(2008).Iconicity of sequence: A corpusbased analysis of the positioning of temporal adverbial clauses in English.Cognitive Linguistics,19,465-490.
10. Eid, M.(Ed.),Ivrson, G.(Ed.)(1993).Principles and prediction: The analysis of natural language.Amsterdam:John Benjamins.
11. Emmorey, K.(Ed.),Lane, H.(Ed.)(2000).The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima.Mahwak, NJ:Lawrence Erlbaum.
12. Fadiga, L.,Fogassi, L.,Pavesi, G.,Rizzolatti, G.(1995).Motor facilitation during action observation: A magnetic stimulation study.Journal of Neurophysiology,73,2608-2611.
13. Gallese, V.,Lakoff, G.(2005).The brain's concepts: The role of the sensory-motor system in conceptual knowledge.Cognitive Neuropsychology,22,455-479.
14. Haiman, J.(1985).Nature syntax: Iconicity and erosion.Cambridge, UK:Cambridge University Press.
15. Harnad, S.(Ed.),Hoest, D.(Ed.),Lancaster, I.(Ed.)(1976).Origins and evolution of language and speech.New York:New York Academy of Sciences.
16. Humphreys, G. W.,Forde, E. M. E.(2001).Hierarchies, similarity, and interactivity in object recognition: On the multiplicity of "category specific" deficits in neuropsychological populations.Behavioral and Brain Sciences,24,453-509.
17. Jescheniak, J. D.,Levelt, W. J.(1994).Word frequency effects in speech production: Retrieval of syntactic information and of phonological form.Journal of Experimental Psychology: Learning, Memory, and Cognition,20,824-843.
18. Jescheniak, J. D.,Meyer, A. S.,Levelt, W. J.(2003).Specific-word frequency is not all that counts in speech production: Comments on Caramazza, Costa, et al. (2001) and new experimental data.Journal of Experimental Psychology: Learning, Memory, and Cognition,29,432-438.
19. Kircher, T.,Straube, B.,Leube, D.,Weis, S.,Sachs, O.,Willmes, K.(2009).Neural interaction of speech and gesture: Differential activations of metaphoric coverbal gestures.Neuropsychologia,47,169-179.
20. Krauss, R. M.,Chen, Y.,Chawla, P.(1996).Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us?.Advances in experimental social psychology,28,389-450.
21. Lai, Y.-D.,Yang, L.-C.(2009).Iconicity and arbitrariness in Taiwan Sign Language: A psycholinguistic account.Mingdao Journal,5,159-187.
22. Lakoff, G.,Johnson, M.(1999).Philosophy in the flesh: The embodied mind and its challenge to western thought.New York:Basic Books.
23. Lin, N.,Guo, Q.,Han, Z.,Bi, Y.(2011).Motor knowledge is one dimension for concept organization: Further evidence from a Chinese semantic dementia case.Brain and Language,119,110-118.
24. Louwerse, M. M.(2008).Embodied relations are encoded in language.Psychonomic Bulletin and Review,15,838-844.
25. Marschark, M.(Ed.),Spencer, P. E.(Ed.)(2003).Oxford handbook of deaf studies, language, and education.New York:Oxford University Press.
26. McNeill, D.(Ed.)(2000).Language and gesture.New York:Cambridge UniversityPress.
27. Parrill, F.,Sweetser, E.(2004).What we mean by meaning: Conceptual integration in gesture analysis and transcription.Gesture,4,197-219.
28. Rauscher, F. H.,Krauss, R. M.,Chen, Y.(1996).Gesture, speech, and lexical access: The role of lexical movements in speech production.Psychological Science,7,226-231.
29. Siakaluk, P. D.,Pexman, P. M.,Sears, C. R.,Wilson, K.,Locheed, K.,Owen, W. J.(2008).The benefits of sensorimotor knowledge: Body-object interaction facilitates semantic processing.Cognitive Science,32,591-605.
30. Simone, R.(Ed.)(1995).Iconicity in language.Amsterdam:John Benjamins.
31. Smith, W. H.,Ting, L. F.(1979).Shou Neng Sheng Chyau [Your hands can become a bridge]: Sign language manual.Taipei, Taiwan:Deaf Sign Language Research Association of the Republic of China.
32. Smith, W. H.,Ting, L. F.(1979).Shou Neng Sheng Chyau [Your hands can become a bridge]: Sign language manual.Taipei, Taiwan:Deaf Sign Language Research Association of the Republic of China.
33. Stefanini, S.,Bello, A.,Caselli, M. C.,Iverson, J. M.,Volterra, V.(2009).Co-speech gestures in a naming task: Developmental data.Language and Cognitive Processes,24,168-189.
34. Tai, J. H.-Y.(2005).Modality effects: Iconicity in Taiwan sign language.POLA FOREVER: Festschrift in celebration of professor William S.-Y. Wang's seventieth birthday,Taipei, Taiwan:
35. Tai, J. H.-Y.(2004).,Chiayi, Taiwan:National Chung Cheng University.
36. Tai, J. H.-Y.(2003).,Chiayi, Taiwan:National Chung Cheng University.
37. Tai, J. H.-Y.(2001).,Chiayi, Taiwan:National Chung Cheng University.
38. Tai, J. H.-Y.(2002).,Chiayi, Taiwan:National Chung Cheng University.
39. Taub, S. F.(2001).Language from the body: Iconicity and metaphor in American Sign Language.Cambridge, UK:Combridge University Press.
40. Thompson, R. L.,Vinson, D. P.,Vigliocco, G.(2009).The link between form and meaning in American Sign Language: Lexical processing effects.Journal of Experimental Psychology: Learning, Memory, and Cognition,35,550-557.
41. Warrington, E. K.,Shallice, T.(1984).Category specific semantic impairments.Brain and Language,107,829-854.
42. Wu, Y. C.,Coulson, S.(2011).Are depictive gestures like pictures? Commonalities and differences in semantic processing.Brain and Language,119,184-195.
43. Zwaan, R. A.,Yaxley, R. H.(2003).Spatial iconicity affects semantic relatedness judgments.Psychonomic Bulletin and Review,10,954-958.