Deep Learning based Loitering Detection System using Multi-camera Video Surveillance Network

Abstract

A deep-learning-based Loitering Detection System (LDS) with re-identification (ReID) capability over a multicamera network is proposed. The proposed LDS is mainly comprising of object detection and tracking, loitering detection, feature extraction, camera switching, and re-identification of the loiterer. The person is detected using You Only Look Once (YOLOv3) and tracked using Simple Online Real-time Tracking with a deep as-sociation matrix (DeepSORT). From the trajectory analysis, once the time and displacements thresholds are satisfied, the person is treated as a loiterer. When the loiterer moves one camera to another, then the algorithm is switched to the appropriate camera feed as per the proposed camera switching algorithm to minimize the computational cost. Subsequently, the loiterer is re-identified in the switched camera feed by comparing the features of the loiterer extracted by the MobileNets with that of the other detected persons based on the triplet loss criteria. The proposed system provides an enhanced accuracy of 96 % on average fps of 33 (without ReID) and 81.5 % at average fps of 30 (with ReID).