一般觀眾對於「類神經網路」之藝術風格轉移認知模式先期研究

A Pilot Study on Audience's Cognitive Model of Neural Style Transfer

呂燕茹(Yanru Lyu)；林伯賢(Po-Hsien Lin)；林榮泰(Rungtai Lin)

認知人因工程 ； 人工智慧 ； 類神經網路 ； 藝術風格轉移 ； Cognitive Ergonomics ； Artificial Intelligent ； Neural Network ； Artistic Style Transfer

設計學報

26卷3期（2021 / 09 / 01）

61 - 84

繁體中文

近年隨著人工智慧所引領的類神經網路深度學習技術日趨成熟，而其相關應用已經逐漸蔓延至包括藝術在內的各個領域，對藝術作品的創作、體驗、審美和欣賞將帶來新的機遇和挑戰。目前該領域多注重演算法的精進，對於演算法的精進主要牽涉到兩個問題：1.就科技（理性）而言，觀眾能否辨認經由電腦的藝術風格轉移？2.就藝術（感性）而言，哪些主要因素會影響經由電腦的藝術風格轉移？因此，本研究透過認知人因工程研究，分析人們如何感知創作者的編碼過程（色彩、筆觸、紋理）與閱聽者的解碼過程（技術、語意、效果）之影響，以期能對於構建人工智慧應用於藝術創作之研究具有積極的助益。本研究係屬一系列的相關研究，先期研究招募了31位藝術、美學與設計等專家學者參與，對野獸派畫作與對應轉換圖的契合度和認知效果進行評估。結果發現，不同繪畫的風格特性是可以被認知與分辨，風格轉移的整體效果會影響觀眾的喜好度；最後，提出一個探討藝術風格轉移的研究模式。除了有助於藝術風格轉移演算法的改進外，也為人工智慧應用於藝術創作相關研究提供參考。

In recent years, the use of AI is becoming more mature with the development of neural network technology, and access to various fields including art creation. Meanwhile, it brings new opportunities and challenges in the field of invention, experience, aesthetics and appreciation of art. At present, this field only focuses on the optimization of algorithms, but there are two key points in the improvement of algorithms. (1) As far as technology (rationality) is concerned, can the audience recognize the artistic style transfer through computers? (2) In terms of Art (sensibility), what are the main factors that affect the artistic style transfer through computers? Therefore, the purpose of this paper is to analyze the difference of audience's cognition during the creator's encoding process (color, stroke, texture) and the audience's decoding process (technical level, semantic level, effectiveness level). This study is a series of related studies, in which 31 experts and scholars with a background in art, aesthetics and/or design, were recruited to participate in the previous study to evaluate the degree of fitness and the effect of cognitive between fauvism portraits and corresponding converted images. The results showed that the stylistic characteristics of different images can be recognized and distinguished by subjects. The overall effect of the style transfer will affect the audience's preferences. At last, a research model of artistic style transfer is put forward. The results of the research not only provide suggestions for the optimization of the NST algorithm but also provide some references for the research on the application of AI into art creation.

1. 汪曼穎, M. Y.,葉怡玉, Y. Y.,黃榮村, J. T.(2013)。臺灣認知心理學的應用：從認知研究到科技脈絡裡的人性化設計。中華心理學刊，55(3)，381-404。
   連結：
2. 曾靖越, C. Y.(2018)。無縫空間的沈浸感：虛擬實境。國教新知，65(3)，105-120。
   連結：
3. Ackoff, R. L.(1989).From data to wisdom.Journal of Applied Systems Analysis,16(1),3-9.
4. Arnheim, R.(1965).Art and visual perception: A psychology of the creative eye.Berkeley, CA:University of California Press.
5. ArtLinks.（2017 年 5 月 18 日）。馬塞爾‧杜尚，挑戰藝術底線的先鋒藝術家。取自：https://kknews.cc/culture/5a3evnl.htmlArtLinks. (2017, May 18). Marcel Duchamp, pioneer artists who challenge the bottom line of art. Retrieved from https://kknews.cc/culture/5a3evnl.html [in Chinese, semantic translation]
6. Augustin, M. D.,Defranceschi, B.,Fuchs, H. K.,Carbon, C.,Hutzler, F.(2011).The neural time course of art perception: An ERP study on the processing of style versus content in art.Neuropsychologia,49(7),2071-2081.
7. Bar, Y.,Levy, N.,Wolf, L.(2015).Classification of artistic styles using binarized features derived from a deep neural network.Computer Vision - ECCV 2014 Workshops. ECCV 2014,Cham:
8. Beardsley, M. C.(1981).Aesthetics, problems in the philosophy of criticism.Indianapolis, IN:Hackett.
9. Beckett, W.(1994).The story of painting.London:DK.
10. Bengio, Y.(2009).Learning deep architectures for AI.Found Trends Mach Learn,2(1),1-127.
11. Benjamin, R. H.(1993).The decorative landscape, fauvism, and the arabesque of observation.The Art Bulletin,75(2),295-316.
12. Chan, C. S.(2015).Style and creativity in design.New York, NY:Springer.
13. Chatterjee, A.(2004).Prospects for a cognitive neuroscience of visual aesthetics.Bulletin of Psychology and the Arts,4,55-59.
14. Chen, C.(2020)。Tokyo Institute of Technology。
15. Choi, T. J.,Ahn, C. W.(2019).Artificial life based on boids model and evolutionary chaotic neural networks for creating artworks.Swarm and Evolutionary Computation,47,80-88.
16. Condorovici, R. G.,Florea, C.,Vrânceanu, R.,Vertan, C.(2013).Perceptually-inspired artistic genre identification system in digitized painting collections.Proceedings of Scandinavian Conference on Image Analysis,Berlin:
17. Conkey, M.(2006).Style, design, and function.Handbook of material culture,London:
18. Djonov, E.,Van Leeuwen, T.(2011).The semiotics of texture: From tactile to visual.Visual Communication,10(4),541-564.
19. Efros, A. A.,Freeman, W. T.(2001).Image quilting for texture synthesis and transfer.Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques,New York, NY:
20. Fiske, J.(2010).Introduction to communication studies.London:Routledge.
21. Gao, Y.,Wu, J.,Lee, S.,Lin, R.(2019).Communication between artist and audience: A case study of creation journey.Proceedings of International Conference on Human-Computer Interaction,Orlando, FL:
22. Gardner, H.(1970).Children’s sensitivity to painting styles.Child Development,41(3),813-821.
23. Gatys, L. A.,Ecker, A. S.,Bethge, M.(2016).Image style transfer using convolutional neural networks.Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition,Los Alamitos, CA:
24. Gatys, L. A.,Ecker, A. S.,Bethge, M.(2015).,未出版
25. Gatys, L. A.,Ecker, A. S.,Bethge, M.,Hertzmann, A.,Shechtman, E.(2017).Controlling perceptual factors in neural style transfer.Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition,Los Alamitos, CA:
26. Getlein, M.,Gilbert, R.(2008).Living with art.New York, NY:McGraw-Hill.
27. Haeberli, P.(1990).Paint by numbers: Abstract image representations.Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques,Dallas, TX:
28. He, B.,Gao, F.,Ma, D.,Shi, B.,Duan, L. Y.(2018).Chipgan: A generative adversarial network for chinese ink wash painting style transfer.Proceedings of the 26th ACM international conference on Multimedia,Seoul:
29. Hertzmann, A. (2018). Image stylization: History and future (Part 3). Retrieved from https://research.adobe.com/news/image-stylization-history-and-future-part-3/
30. Hoquet, T.(2005).Buffon: histoire naturelle et philosophie.Paris:Honoré Champion.
31. Huang, X.,Belongie, S.(2017).Arbitrary style transfer in real-time with adaptive instance normalization.Proceedings of the IEEE International Conference on Computer Vision,Venice:
32. Jakobson, R.,Pomorska, K.,Rudy, S.(1987).Language in literature.Cambridge, MA:Harvard University Press.
33. Jing, Y.,Yang, Y.,Feng, Z.,Ye, J.,Yu, Y.,Song, M.(2020).Neural style transfer: A review.IEEE Transactions on Visualization and Computer Graphics,26(11),3365-3385.
34. Johnson, J.,Alahi, A.,Li, F.(2016).Perceptual losses for real-time style transfer and super-resolution.Proceedings of 2016 European conference on computer vision,Amsterdam:
35. Joshi, B.,Stewart, K.,Shapiro, D.(2017).Bringing impressionism to life with neural style transfer in come swim.Proceedings of the ACM SIGGRAPH Digital Production Symposium,Los Angeles, CA:
36. Karayev, S.,Trentacoste, M.,Han, H.,Agarwala, A.,Darrell, T.,Hertzmann, A.,Winnemoeller, H.(2013).,未出版
37. Kurzweil, R.(2005).The singularity is near: When humans transcend biology.London:Penguin.
38. Lakoff, G.,Johnson, M.(1980).Metaphors we live by.Chicago, IL:Chicago University Press.
39. Lee, S. G.,Cha, E. Y.(2016).Style classification and visualization of art painting’s genre using selforganizing maps.Human-Centric Computing and Information Sciences,6(1),7.
40. Li, Y.,Fang, C.,Yang, J.,Wang, Z.,Lu, X.,Yang, M. H.(2017).,未出版
41. Lin, R.(1992).An application of the semantic differential to icon design.Proceedings of the Human Factors Society Annual Meeting,Los Angeles, CA:
42. Lin, R.,Kreifeldt, J. G.(2001).Ergonomics in wearable computer design.International Journal of Industrial Ergonomics,27(4),259-269.
43. Liu, L.,Xi, Z.,Ji, R.,Ma, W.(2019).Advanced deep learning techniques for image style transfer: A survey.Signal Processing: Image Communication,78,465-470.
44. Mazzone, M.,Elgammal, A.(2019).Art, creativity, and the potential of artificial intelligence.Arts,8(1),26.
45. McGuinness, D. E.(2016).Painting: Materials, techniques, styles, and practice.New York, NY:Rosen Education Service.
46. Mishory, A.(2000).Art history: An introduction.Ra'anana:Open University of Israel.
47. Mishra, A. K.(2018).A DIKW architecture for cognitive engineering.Procedia Computer Science,123,285-289.
48. Schumaker, R. P.(2011).From data to wisdom: The progression of computational learning in text mining.Communications of the IIMA,11(1),4.
49. Shamir, L.,Macura, T.,Orlov, N.,Eckley, D. M.,Goldberg, I. G.(2010).Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art.ACM Transactions on Applied Perception (TAP),7(2),1-17.
50. Silverman, K.(1984).The subject of semiotics.New York, NY:Oxford University Press.
51. Sims, K.(1991).Artificial evolution for computer graphics.ACM SIGGRAPH Computer Graphics,25(4),319-328.
52. Spehr, M.,Wallraven, C.,Fleming, R. W.(2009).Image statistics for clustering paintings according to their visual appearance.Proceedings of Computational Aesthetics 2009: Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging,Aire-La-Ville, Switzerland:
53. Stangos, N. (Ed.)(1994).Concepts of modern art: From Fauvism to Postmodernism.London:Thames & Hudson.
54. Steenberg, E.(2007).Visual aesthetic experience.The Journal of Aesthetic Education,41(2),89-94.
55. Taylor, J.,Witt, J.,Grimaldi, P.(2012).Uncovering the connection between artist and audience: Viewing painted brushstrokes evokes corresponding action representations in the observer.Cognition,125(1),26-36.
56. Zeleny, M.(2005).Human systems management: Integrating knowledge, management and systems.Singapore:World Scientific.
57. Zhang, Y.,Fang, C.,Wang, Y.,Wang, Z.,Lin, Z.,Fu, Y.,Yang, J.(2019).Multimodal style transfer viagraph cuts.Proceedings of 2019 IEEE/CVF International Conference on Computer Vision,Seoul:
58. Zujovic, J.,Gandy, L.,Friedman, S.,Pardo, B.,Pappas, T. N.(2009).Classifying paintings by artistic genre: An analysis of features & classifiers.Proceedings of 2009 IEEE International Workshop on Multimedia Signal Processing,Rio de Janeiro, Brazil:
59. 林榮泰（2003 年 8 月 7 日）。人與機器的對話：科技始終來自於人性？取自：https://scitechvista.nat.gov.tw/c/sWUc.htmLin, R. (2003, August 7). Dialogue between human and machine: Does technology always comes from humanity? Retrieved from https://scitechvista.nat.gov.tw/c/sWUc.htm [in Chinese, semantic translation]
60. 林榮泰, R.(2007)。蒙家就是我家－從蒙得裡安談文化創意的簡約設計。藝術欣賞，3(5)，4-9。
61. 林榮泰, R.,李仙美, S.(2015).詩情畫意─仙雲之美習作經驗分享.新北市=New Taipei City:國立臺灣藝術大學=National Taiwan University of Arts.
62. 洪銘駿, M. J.(2016)。臺北市=Taipei, Taiwan，國立臺灣大學=National Taiwan University。
63. 戚良德,劉勰, X.(2008).文心雕龍校注通譯.上海市=Shanghai:上海古籍出版社=Shanghai Classics Publishing House.
64. 許素朱（2019b）。計畫簡介。取自：https://techart.nthu.edu.tw/ai/?page_id=32Hsu, S. C. (2019b). Introduction to the plan. Retrieved from https://techart.nthu.edu.tw/ai/?page_id=32 [in Chinese, semantic translation]
65. 許素朱, S. C.(2019)。人工智慧與藝術的競合創作。藝術家，535，240-245。
66. 量子位（2017）。AI 通過了藝術創作圖靈測試，機器發展出了自己的美感。取自https://kknews.cc/news/b48zovj.htmlQbitai. (2017). AI passed the turing test of artistic creation, and the machine developed its own beauty. Retrieved from https://kknews.cc/news/b48zovj.html [in Chinese, semantic translation]
67. 葉衛平, W. P.(2018).藝術的內在發生.北京市=Beijing:清華大學出版社=Tsinghua University Press.
68. 劉育成（2019 年 6 月 19 日）。機器學習（人工智慧）如何可能重新定義「創作」：「模糊性」作為創造性的來源之一。取自：https://www.digiarts.org.tw/DigiArts/DataBasePage/4_140864213871031/ChiLiu, Y. C. (2019, June 19). How can machine learning (AI) can redefine “creation”: “Fuzziness” as a source of creativity. Retrieved from https://www.digiarts.org.tw/DigiArts/DataBasePage/4_140864213871031/Chi [in Chinese, semantic translation]
69. 謝其昌, C. C.(2015)。論油畫的材料發展談繪畫技法的形式與演變。美學與視覺藝術學刊，7，35-50。

1. (2024)。高中生對校園公共藝術美感偏好之研究－以國立臺北大學三峽校區教學大樓新建工程公共藝術設置計畫為例。高雄師大學報：人文與藝術類，56，21-40。