Emotion Recognition Using Facial Thermal Imaging: A Comparative Study

Abstract

This paper investigates thermal facial emotion recognition using classical statistical methods and deep learning. Thermal imaging captures facial infrared radiation, offering a privacy-preserving and illumination-invariant alternative to visible-light imaging. We reproduce classical approaches from the KTFE dataset literature — Principal Component Analysis (PCA), the Eigenspace Minimum Classifier (EMC), and their combined PCA–EMC framework — and compare them against VGG-16. Experiments are conducted on 2140 images across six emotion classes from the KTFE database. VGG-16, pretrained on ImageNet and adapted with a Dense classification head and dropout regularization, is trained on 128 × 128 thermal facial images using the Adam optimizer with a frozen convo-lutional backbone. VGG-16 achieves classification accuracies up to 99.90%, substantially outperforming all classical baselines and confirming that deep convolutional networks can effectively learn discriminative thermal facial patterns. These results highlight the strong potential of thermal imaging for accurate, robust, and privacy-aware emotion recognition in human–computer interaction, mental health monitoring, and security applications.