結合體感與注意力辨識之情感運算研究-以3D校園導覽為例

The Affective Computing Study of Posture and Attention Recognition - An Example on The 3D School Guide System

林千慧

情感運算 ； 導覽系統 ； 注意力辨識 ； 肢體辨識 ； Affective Computing ； guide system ； attention recognition ； posture recognition

臺中科技大學資訊管理系碩士班學位論文

2016年

碩士

吳明霓

繁體中文

近年來情感運算(Affective Computing)的興起，已徹底改變了以往人機互動(Human-computer Interaction)的思考設計模式。為了讓人類與電腦之間的互動機制能更加的直覺與人性化，如今越來越多的研究將人類的情緒行為視為人機互動的關鍵因素，並嘗試將情感運算導入到系統中運用。因此，本研究希望將肢體動作和眼動行為納入人類情緒表達的因素，並嘗試將肢體情緒及注意力情緒結合到依據真實校園所開發的3D校園導覽系統。希望透過使用者在遊覽校園環境的過程中，分析使用者肢體的行為與視線的注視來作為情緒判斷的依據，並顯示對應區域與情緒的回饋內容。透過這樣的方式，以減少操作上的負荷，改善使用者的互動體驗，達到直覺、人性化的自然感受。 本研究主要開發了兩種結合不同情緒的操控模式，分別為肢體情緒操控與注意力情緒操控。為探討是否加入情感因素的操作方式能改善以往的互動感受，本研究另外將鍵盤操控和體感操控模式一併納入比較。結果顯示，大部分受測者對於結合情感辨識的操作在整體的互動感受上是滿意的，並且能顯著降低操作上的負荷，也認為與以往的操作模式相比較是有趣的。最後，本研究也列出研究上的限制與未來可以改進的方向。

The Affective Computing has changed the design thinking patterns of the Human-computer Interaction completely in recent years. In order to promote the intuitive and user-friendly on Human-Computer Interaction (HCI) system, there are more and more studies regard human emotions and behaviors as a key element in HCI, and try to introduce affective computing into system. Therefore, this study hopes to combine body movement and eyes gaze behavior into the system, and try to integrate emotion expression in posture and attention into the 3D School Guide System which developed on real campus. This study hope to analyze user’s body movement and gaze as the emotional judgment, and shows content feedback correspond to the areas and the emotions through the process of users are visiting the school environment. In this way, the operation burden can be reduced and user's interactive experience can be improved to achieve intuitive and user-friendly experience. This study developed two different emotion operational modes, the operation of body emotion and attention emotion. In order to investigate whether to join the operation of emotional factors can improve the past interactive experience, keyboard operation and motion sensing operation were incorporated in comparison. The results in the overall interactive experience showed that most of the participants were satisfied with the operation modes which integrated emotion recognition, and not only reduced operating load significantly, but also thought that they are more interesting than the past operation modes. Finally, this study showed the limits of this study and improving policies for future studies.

1. [1] Bergstrom, J. R., & Schall, A. (Eds.). (2014). Eye tracking in user experience design. Elsevier.
   連結：
2. [2] Buscher, G., Cutrell, E., & Morris, M. R. (2009). What do you see when you're surfing?: using eye tracking to predict salient regions of web pages. In Proceedings of the SIGCHI conference on human factors in computing systems, 21-30. ACM.
   連結：
3. [3] Chen, Y., & Tsai, M. J. (2015). Eye-hand coordination strategies during active video game playing: An eye-tracking study. Computers in Human Behavior, 51, 8-14.
   連結：
4. [4] Coulson, M. (2004). Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. Journal of nonverbal behavior, 28(2), 117-139.
   連結：
5. [5] Dael, N., Mortillaro, M., & Scherer, K. R. (2012). Emotion expression in body action and posture. Emotion, 12(5), 1085.
   連結：
6. [6] De Meijer, M. (1989). The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior, 13(4), 247-268.
   連結：
7. [7] D'Mello, S., Olney, A., Williams, C., & Hays, P. (2012). Gaze tutor: A gaze-reactive intelligent tutoring system. International Journal of Human-computer Studies, 70(5), 377-398.
   連結：
8. [10] Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., & Scherer, K. (2011). Toward a minimal representation of affective gestures. Affective Computing, IEEE Transactions on, 2(2), 106-118.
   連結：
9. [11] Irwin, D. E. (1992). Memory for position and identity across eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(2), 307.
   連結：
10. [13] Juslin, P. N., & Sloboda, J. (Eds.). (2011). Handbook of music and emotion: Theory, research, applications. Oxford University Press.
    連結：
11. [14] Kleinsmith, A., & Bianchi-Berthouze, N. (2013). Affective body expression perception and recognition: A survey. Affective Computing, IEEE Transactions on, 4(1), 15-33.
    連結：
12. [15] Land, M. F., & Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities?. Vision Research, 41(25), 3559-3565.
    連結：
13. [16] Lee, H., Lim, S. Y., Lee, I., Cha, J., Cho, D. C., & Cho, S. (2013). Multi-modal user interaction method based on gaze tracking and gesture recognition. Signal Processing: Image Communication, 28(2), 114-126.
    連結：
14. [18] Picard, R. W. (1997). Affective computing (Vol. 252). Cambridge: MIT press.
    連結：
15. [19] Piqueras-Fiszman, B., Velasco, C., Salgado-Montejo, A., & Spence, C. (2013). Using combined eye tracking and word association in order to assess novel packaging solutions: A case study involving jam jars. Food Quality and Preference, 28(1), 328-338.
    連結：
16. [20] Ramani, K. (2015). A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery. Computer-Aided Design, 69, 11-24.
    連結：
17. [21] Rizzolatti, G., Riggio, L., Dascola, I., & Umiltá, C. (1987). Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention. Neuropsychologia, 25(1), 31-40.
    連結：
18. [22] Roupé, M., Bosch-Sijtsema, P., & Johansson, M. (2014). Interactive navigation interface for virtual reality using the human body. Computers, Environment and Urban Systems, 43, 42-50.
    連結：
19. [23] Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161.
    連結：
20. [24] Schouwstra, S. J., & Hoogstraten, J. (1995). Head position and spinal position as determinants of perceived emotional state. Perceptual and Motor Skills, 81(2), 673-674.
    連結：
21. [25] Thayer, R. E. (1989). The biopsychology of mood and arousal. Oxford University Press.
    連結：
22. [27] Thompson, N., & McGill, T. J. Affective stack — A model for affective computing application development. Journal of Software, 10(8), 919-930.
    連結：
23. [28] Vu, T. M. H., Tu, V. P., & Duerrschmid, K. (2015). Design factors influence consumers’ gazing behaviour and decision time in an eye-tracking test: A study on food images. Food Quality and Preferenc, 47, 130-138.
    連結：
24. [29] Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98(2), 219.
    連結：
25. [30] Yarbus, A. L. (1967), Eye movements and vision. Springer US.
    連結：
26. [31] Zhang, B., & Seo, H. S. (2015). Visual attention toward food-item images can vary as a function of background saliency and culture: An eye-tracking study. Food Quality and Preference, 41, 172-179.
    連結：
27. [32] 陳廷暐. (2011). 基於體感互動科技發展互動式溝通與情緒感知平台. 元智大學機械工程研究所碩士論文。
    連結：
28. [8] Duchowski, A. (2007). Eye tracking methodology: Theory and practice (Vol. 373). Springer Science & Business Media.
29. [9] Ekman, P., & Friesen, W. V. (2003). Unmasking the face: A guide to recognizing emotions from facial clues. Ishk.
30. [12] Jiménez, J. (2014). Application of eye-tracking methodology for fragrance evaluation. Scientific Papers, 23.
31. [17] Michailidou, E., Harper, S., & Bechhofer, S. (2008, September). Visual complexity and aesthetic perception of web pages. In Proceedings of the 26th annual ACM international conference on Design of communication, 215-224. ACM.
32. [26] Tomkins, S. S. (1984). Affect theory. Approaches to Emotion, 163, 195.
33. [33] 譚發銘. (2014). 虛擬校園導覽系統設計與實作-以亞洲大學為例. 亞洲大學資訊傳播學系碩士論文。
34. [34] 魏佑嘉. (2014). 使用 Kinect 之直覺式 3D 博物館導覽系統. 真理大學資訊工程學系碩士班碩士論文。
35. [35] Eye tracking. (n.d.). Retrieved October 20, 2015, from https://en.wikipedia.org/wiki/Eye_tracking
36. [36] Getting ready for shipping. (n.d.). Retrieved November 7, 2015, from https://en.wikipedia.org/wiki/Eye_tracking
37. [37] 虛擬實境（無日期）。2015年10月29日，取自： https://en.wikipedia.org/wiki/Eye_tracking
38. [38] 情感交互—移动应用产品交互趋势（2013年2月6日）。2015年10月25日，取自： http://www.cocoachina.com/programmer/20130206/5660.html
39. [39] 情感運算（2012年4月12日）。2015年10月25日，取自： https://selinachang2581.wordpress.com/2012/04/12/%E6%83%85%E6%84%9F%E9%81%8B%E7%AE%97/
40. [40] Kinect 應用程式開發入門（無日期）。2015年11月06日，取自： https://msdn.microsoft.com/zh-tw/hh367958.aspx