A Novel Deep Learning Framework with Behavioral Heatmap, for Malware Classification

Abstract

The classification of malware continues to be a significant cybersecurity problem because malicious software can rapidly evolve obfuscation and evasion mechanism to disguise its real behaviour. Common static analysis techniques can often fail to detect malware that is polymorphic or metamorphic, meaning that code structure changes but malicious intent remain the same. This paper proposes a dynamic behaviour-based malware classification framework as a solution to these issues. It generates visually understandable Custom RGB (Infernoinspired) heatmap images by using API call sequences and argument features captured during execution. These perceptually consistent heatmaps, which transition from deep blue tomagenta to bright yellow, enhance the visual separability of subtle behavioural differences across malware families. Three deep learning architectures are implemented and evaluated: Existing CNN Architecture, Proposed EfficientNet Model and Proposed CNN Model that employed transfer learning, for finegrained feature extraction. The accuracy results for tests on 22,056 samples dataset from eight malware families were 98.76%, 98.88% and 99.12% respectively. The results demonstrate how behavioural visualisation, in conjunction with hybrid and multistage deep learning architectures, significantly enhances malware classification accuracy and resilience to obfuscation-based attacks. This approach provides a vision-driven, interpretable approach for cybersecurity systems of the future.