Face Expression Recognition based on Attention-lightweight Convolutional Neural Networks

10.6919/ICJE.202205_8(5).0032

Pengzhan Feng

Facial Emotion Recognition ； Convolutional Neural Networks ； Depthwise Separable Convolutions ； Spatial Attention ； Channel Attention

International Core Journal of Engineering

8卷5期（2022 / 05 / 01）

268 - 278

英文

For the current problems of complex parameters of face expression recognition network models and feature redundancy arising from network training, an improved method of lightweight VGGNet network is proposed. Firstly, using the VGGNet network structure, the depth-separable convolution and global average pool are introduced into the original network to reduce the network parameters; secondly, an attention mechanism with spatial attention and channel attention in parallel is designed, and the attention module is embedded in front of the pooling layer and for feature reconstruction to enhance the apparent deep feature information and suppress the useless feature information; finally, the extracted features are fed into the classifier to realize face expression classification. Experiments are conducted and analyzed on CK+ and RaDF databases. The recognition accuracy of 98.86% was tested on the CK+ dataset containing only frontal face images, and 97.48% was tested on the RaDF database of multi-pose face expression images, which proved the effectiveness of this improved network for multi-pose facial expression recognition.

1. Niu B, Gao Z, Guo B. Facial Expression Recognition with LBP and ORB Features [J]. HERNÁNDEZ-PÉREZ J A. Computational Intelligence and Neuroscience, 2021, 2021 : 1–10.
   連結：
2. Rao Q, Qu X, Mao Q, et al. Multi-pose facial expression recognition based on SURF boosting [C]//2015 International Conference on Affective Computing and Intelligent Interaction (ACII). Xi’an, China:, IEEE, 2015 : 630–635.
   連結：
3. Dagher I, Dahdah E, Al Shakik M. Facial expression recognition using three-stage support vector machines[J]. Visual Computing for Industry, Biomedicine and Art, 2019, 2 : 24.
   連結：
4. Islam B, Mahmud F, Hossain A, et al . Human Facial Expression Recognition System Using Artificial Neural Network Classification of Gabor Feature Based Facial Expression Information[C]//2018 4th International Conference on Electrical Engineering and Information Communication Technology (iCEEiCT). 2018 : 364–368.
   連結：
5. Zavarez M V, Berriel R F, Oliveira-Santos T. Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network [C]//2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). Niteroi:, IEEE, 2017 : 405–412.
   連結：
6. Akhand M A H, Roy S, Siddique N, et al. Facial Emotion Recognition Using Transfer Learning in the Deep CNN [J]. Electronics, 2021, 10(9) : 1036.
   連結：
7. Roy A G, Navab N, Wachinger C. Recalibrating Fully Convolutional Networks With Spatial and Channel “Squeeze and Excitation” Blocks [J]. IEEE Transactions on Medical Imaging, 2019, 38(2) : 540–549.
   連結：
8. Wang Q, Wu B, Zhu P, et al. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA:, IEEE, 2020 : 11531–11539.
   連結：
9. Lucey P, Cohn J F, Kanade T, et al. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression [C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. San Francisco, CA, USA:, IEEE, 2010 : 94–101.
   連結：
10. Langner O, Dotsch R, Bijlstra G, et al. Presentation and validation of the Radboud Faces Database [J]. Cognition & Emotion, 2010, 24(8) : 1377–1388.
    連結：
11. Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI:, IEEE, 2017 : 1800–1807.
    連結：
12. Woo S, Park J, Lee J-Y, et al. CBAM: Convolutional Block Attention Module [G]//FERRARI V, HEBERT M, SMINCHISESCU C, et al. Computer Vision – ECCV 2018. Cham:, Springer International Publishing, 2018, 11211 : 3–19.
    連結：
13. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. arXiv:1409.1556 [cs], 2015.
14. Pramerdorfer C, Kampel M. Facial Expression Recognition using Convolutional Neural Networks: State of the Art[J]. 2016Facial Expression Recognition using Convolutional Neural Networks.
15. Howard A G, Zhu M, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[J]. arXiv:1704.04861 [cs], 2017MobileNets.
16. Lin M, Chen Q, Yan S. Network In Network [J]. arXiv:1312.4400 [cs], 2014.
17. Huang G, Liu Z, Van Der Maaten L, et al. Densely Connected Convolutional Networks [J]. arXiv:1608. 06993 [cs], 2018.
18. Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation Networks [J]. arXiv:1709.01507 [cs], 2019.