Mask RCNN-based Single Shot Multibox Detector For Gesture Recognition In Physical Education

10.6180/jase.202303_26(3).0009

Tao Feng

Human-computer interaction ； gesture recognition ； mask RCNN ； single shot multibox detector ； loss function ； physical education

淡江理工學刊

26卷3期（2023 / 03 / 01）

377 - 385

英文

Human-computer interaction (HCI) is an important supporting technology in the computer vision area, especially in physical education. HCI can promote the efficiency of physical education class, which is of great help to improve the learning efficiency. It is developing towards naturalization, intelligence, high efficiency, and materialization. Gesture recognition is very important in HCI, and plays a very important role in artistic understanding and image perception. Traditional gesture recognition methods are prone to misrecognition and result in low accuracy. In this paper, we propose a new gesture recognition method based on mask RCNN and single shot multibox detector (SSD) in HCI. Firstly, feature extraction and region segmentation are performed on the red, green, and blue (RGB) three-channel images, and the hand instance segmentation and mask are obtained. Then we modify the SSD model to obtain a new convolution layer, which can realize the fusion of shallow visual convolution layer and deep semantic convolution layer in the network structure. To solve the problem of poor classification performance caused by the imbalance of positive and negative samples, an improved loss function is proposed to improve the model ability of classifying target gestures. The experimental results show that compared with state-of-the-art methods, the proposed method has better robustness and faster detection speed while maintaining higher gesture detection accuracy.