Can I read Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning on EtoBox?

Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning by Yeong-Hyeon Byeon; Dohyung Kim; Jaeyeon Lee; Keun-Chang Kwak is a Engineering article available to read on EtoBox.

What is Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning about?

Behavior recognition has applications in automatic crime monitoring, automatic sports video analysis, and context awareness of so-called silver robots. In this study, we employ deep learning to recognize behavior based on body and hand–object interaction regions of interest (ROIs). We propose an ROI-based four-stream ensemble convolutional neural network (CNN). Behavior recognition data are mainly composed of images and skeletons. The first stream uses a pre-trained 2D-CNN by converting the 3D skeleton sequence into pose evolution images (PEIs). The second stream inputs the RGB video into the 3D-CNN to extract temporal and spatial features. The most important information in behavior recognition is identification of the person performing the action. Therefore, if the neural network is trained by removing ambient noise and placing the ROI on the person, feature analysis can be performed by focusing on the behavior itself rather than learning the entire region. Therefore, the third stream inputs the RGB video limited to the body-ROI into the 3D-CNN. The fourth stream inputs the RGB video limited to ROIs of hand–object interactions into the 3D-CNN. Finally, because better performance i

Who reads Body and Hand–Object ROI-Based Behavior Recognition Using Deep Learning?

Author

Yeong-Hyeon Byeon; Dohyung Kim; Jaeyeon Lee; Keun-Chang Kwak

Publisher

MDPI AG

Published

2021

Language

EN

Field

Engineering (Physical Sciences)