The Application of Facial Expression Recognition to the Evaluation of Stylized Images

臉部表情辨識應用於式樣化圖像評估

10.30105/JDS.201507_18(1).0005

余佳穎(Chia-Yin Yu)；柯志祥(Chih-Hsiang Ko)

臉部表情辨識 ； 視覺圖像 ； 式樣化圖像 ； Facial Expression Recognition ； Visual Image ； Stylized Image ； FaceReader

設計學研究

18卷1期（2015 / 07 / 01）

99 - 122

英文

在商業設計領域中，視覺圖像形式的呈現，可以帶給消費者不同程度的視覺刺激，進而對設計內容或商品產生興趣，誘發購買行為。傳統的臉部表情辨識須採用人工分析，不適用於具時效性與量化研究。FaceReader是一套自動化臉部表情辨識軟體，可以克服傳統臉部表情辨識的限制，客觀且即時分析受測者情緒。本研究以視覺圖像為實驗樣本，根據相關文獻，將樣本依照式樣化程度分成圖像插畫、圖形描繪、圖形符號、抽象符號四類，以臉部表情辨識為基礎，架構情緒辨識實驗，了解受測者對不同式樣化程度的圖像，是否具有情緒差異。在使用FaceReader辨識120位受測者情緒後，進行重複量數單因子變異數分析，並且與訪談內容互相對照。研究結果顯示受測者對式樣化程度不同之圖像樣本，在「快樂」、「憤怒」、「驚訝」、「厭惡」情緒上，具顯著性差異。本研究證實FaceReader應用於商業設計中領域中，評估受測者對視覺圖像的情緒之可行性與實用性。然而，鑒於人類情緒的複雜性，研究者建議在使用FaceReader客觀取得受測者情緒外，仍需輔以觀察報告或受測者訪談，方能了解受測者的情緒內涵與可能產生的原因。而本研究探討潛在消費族群對不同程度式樣化圖像的情緒認知，亦可協助相關領域於設計實務、市場行銷，作為參考與依據。

In the field of commercial design, the presentation of visual images can provide consumers with various visual stimuli to arouse their interests in design or products and to induce purchase behavior. Traditional facial expression recognition is done by manual analysis, which is time consuming and is not suitable for quantitative research. FaceReader is an automatic facial expression recognition software, which can be used to overcome the limit of traditional facial expression recognition and to analyze subjects' emotions objectively and instantly. Visual images were used as samples in this research and were categorized according to relevant literature by the states of stylization into four types: pictorial illustration, graphic rendering, graphic symbology and abstract symbology. An emotional recognition experiment based on facial expression recognition was conducted to understand the subjects' emotional differences in different types of stylized images. FaceReader was used to recognize 120 subjects' emotions by repeated measures one-way ANOVA and the results were compared with the context of individual interviews. The results indicated that there were significant differences in happiness, anger, surprise and disgust in the subjects' emotions toward different stylized images. This research confirms the feasibility and practicability of the application of FaceReader in the evaluation of the subjects' emotions to visual images in the field of commercial design. However, due to the complexity of human emotions, it is suggested that in addition to the objective use of FaceReader for subjects' emotions, supplementary observation loggings or subject's interviews are necessary in order to understand reasons behind subjects' emotional context. It is expected that the exploration of potential consumer groups' emotional cognitions to different stylized images can be used as a reference and basis in relevant fields of practical design and marketing.

1. Berlyne, D. E.(1971).Aesthetiics and Psychobiology.New York:Meredith Corporation.
2. Brunswik, E.(1955).Representative design and probabilistic theory in a functional psychology.Psychological Review,62(3),193-217.
3. da Pos, O.,Green-Armytage, P.(2007).Facial expression, colors and basic emotions.Color: Design & Creativity,1(1),1-20.
4. Danner, L.,Haindl, S.,Joechl, M.,Duerrschmid, K.(2014).Facial expressions and autonomous nervous system responses elicited by tasting different juices.Food Research International,64(10),81-90.
5. D'Arcey, T.,Johnson, M.,Ennis, M.(2012).Assessing the validity of FaceReader using facial electromyography.Proceedings of APS 24th annual meeting,Chicago, IL:
6. de Wijk, R. A.,He, W.,Mensink, M. G. J.,Verhoeven, R. H. G.,de Graaf, C.(2014).ANS responses and facial expressions differentiate between the taste of commercial breakfast drinks.PloS one,9(4),e93823.
7. Ekman, P.(2003).Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life.New York:Times Books.
8. Ekman, P.,Friesen, W. V.(1978).Facial Action Coding System: A Technique for the Measurement of Facial Movement.Palo Also:Consulting Psychologists Press.
9. Ekman, P.,Friesen, W. V.(1969).The repertoire of nonverbal behavior: Categories, origins, usage, and coding.Semiotica,1(1),49-98.
10. Ekman, P.,Friesen, W. V.,Hager, J. C.(2002).Facial Action Coding System: Investigator's Guide.Salt Lake City:Research Nexus.
11. Ekman, P.,Friesen, W. V.,Hager, J. C.(2002).Facial Action Coding System: The Manual.Salt Lake City:Research Nexus.
12. Essa, I. A.,Pentland, A. P.(1997).Coding, analysis, interpretation, and recognition of facial expressions.IEEE Transactions on Pattern Analysis and Machine Intelligence,19(7),757-763.
13. Gatti, A.(2014).Milan, Italy,Department of Design, Politecnico di Milano.
14. Goldie, P. (Ed.)(2002).Understanding Emotions: Mind and Morals.Farnham, Surrey:Ashgate.
15. Hill, D.(2010).Emotionomics: Leveraging Emotions for Business Success.London:Kogan Page.
16. Izard, C.(1971).The Face of Emotion.New York, NY:Appleton-Century-Crofts.
17. Jones, S. S.,Collins, K.,Hong, H.(1991).An audience effect in smile production in 10 month old infants.Psychological Science,2(1),45-49.
18. Lajevardi, S. M.,Wu, H. R.(2012).Facial expression recognition in perceptual color space.IEEE Transactions on Image Processing,21(8),3721-3733.
19. Loijens, L.,Krips, O.(2013).FaceReader Methodology.Wageningen, Nederland:Noldus Information Technology.
20. Meyer, R. P.,Laveson, J. I.(1981).An experience-judgment approach to tactical flight training.Proceedings of the Human Factors Society 25th Annual Meeting,Santa Monica, CA:
21. Motterlini, M.(2008).Economia Emotiva: Che Cosa Si Nasconde Dietro I Nostri Conti Quotidiani.Milano:Rizzoli.
22. Parncutt, R.(Ed.),McPherson, G.(Ed.)(2002).The Science and Psychology of Music Performance: Creative Strategies for Teaching and Learning.Oxford:Oxford University Press.
23. Planalp, S.(1999).Communicating Emotion: Social, Moral, and Cultural Processes.New York:Cambridge University Press.
24. Polgar, S.,Thomas, S. A.(1991).Introduction to Research in the Health Science.Melbourne, Australia:Churchill Livingstone.
25. Scherer, K. R.,Clark-Polner, E.,Mortillaro, M.(2011).In the eye of the beholder? Universality and cultural specificity in the expression and perception of emotion.International Journal of Psychology,46(6),401-435.
26. Smets, N. J. J. M.,Neerincx, M. A.,Looije, R.(2012).Measuring user behavior in a complex USAR team evaluation.Proceedings of Measuring Behavior 2012: 8th International Conference on Methods and Techniques in Behavioral Research,Wageningen, Nederland:
27. Stewart, D. W.,Shamdasani, P. N.(1990).Focus groups: Theory and practice.Newbury Park, CA:Sage.
28. Zaman, B.,Shrimpton-Smith, T.(2006).The FaceReader: Measuring Instant Fun of Use.Proceedings of the 4th Nordic conference on Human-computer interaction: Changing roles,New York:
29. 林榮泰(1993)。評估圖像符號方法的研究。明志工專學報，25，239-256。
30. 胡幼慧(1996)。質性研究理論、方法及本土女性研究實例。臺北市:巨流。
31. 馬漢彥(1993)。十二生肖文化叢書：龍。新北市:尋根圖書。
32. 游韻馨譯、視覺デザイン研究所(2012)。日本、中國傳統紋樣符號：花布、染織、陶瓷、器物的東洋文化解析。臺北市:尖端。
33. 劉奕岑(2012)。情緒與情緒表情符號的關連性研究。樹德科技大學學報，14(1)，185-216。