A Three Stream Deep Network on Extracted Projected Planes for Human Action Recognition

Abstract

Human actions are challenging to recognize as it varies its shape from different angle of perception. To tackle this challenge, a multi view camera set up can be arranged, however, it is not cost effective. To handle this issue, a multi stream deep learning network is proposed in this work which is trained on different 3D projected planes. The extracted projected planes which represents different angle of perception, are used as an alternative to multi view action recognition. The projected planes are such that they represents top, side and front view for the action videos. The projected planes are then fed to a three stream deep convolutional neural network. The network uses transfer learning technique to avoid training from scratch. Finally, the scores from three streams are fused to provide the final score to recognize the query video. To evaluate the proposed work, the challenging KTH dataset is used which is widely used and publicly available. The results show that the proposed work performs better compared to state-of-the-art techniques.