Detecting Image Manipulation in Lossy Compression: A Multi-modality Deep-Learning Framework

Abstract

Due to the advancement of photo editing techniques, it has become easier to create fake photos that look incredibly realistic and are edited in a way that leaves no visible signs of manipulation, making them ideal for synthesis. However, Instagram, WeChat, and TikTok are some of the popular social media platforms where the images have been lossy compressed before uploading them. As a result, learning to spot forged images in their compressed form is crucial. As part of this, some forensic detection techniques have made great strides in uncompressed scenarios, but there is still much to learn about the forensics of lossy compressed images. Therefore, this research proposes a hybrid deep learning framework by dissecting compressed and manipulated images at the preprocessing and feature extraction levels. The suggested noise stream progressively prunes the texture information to prevent the model from fitting the compression noise. Hence, a noise stream is employed to extract temporal correlation characteristics to address the potential problem of ignoring temporal consistency in lossy compressed images. Further, residuals from two streams are fed to custom ResNet blocks to enhance the clues of manipulation and pooled to concatenate the enhanced fingerprints. Finally, the proposed method outperforms state-of-the-art techniques in identifying manipulation in lossy compressed images.