Space-Time Attention Model-based Anomalous Event Classification for Smart City Applications

Abstract

Anomalous event classification automatically identifies anomalous events using the videos in an intelligent video surveillance system. However, anomalous event classification is challenging due to inherent research challenges such as the requirement of high-end computational infrastructure, data imbalances, and data scarcity. Typically, a combination of Convolution Neural Networks (CNNs) and Long-Short-Term-Memory (LSTM) are used to model the spatiotemporal dynamics of the videos for video classification. However, these models have no attention mechanism to boost the relevant spatiotemporal features and discard the irrelevant features. Hence, a Space-Time Attention Model (STAM)-based anomalous event classifier is proposed. The model is trained and validated on the “Anomalous Event Classification 22,” i.e., the “AEC22 dataset” comprising twenty-two anomalous event classes such as abuse, arrest, arson, assault, etc. The STAM is a combined spatial and temporal transformer that takes a series of frames extracted from the input video and predicts corresponding video-level classification as the output. Subsequently, the proposed model provides 92.84% classification accuracy, which is compared with the two stateof-the-art video classification methods to validate its superiority. The proposed model has huge potential for classifying anomalous events in smart city applications.