Klasifikasi serangan malware berbasis citra menggunakan algoritma YOLOv11
DOI:
https://doi.org/10.24246/itexplore.v5i2.2026.pp232-240Keywords:
Malware, YOLOv11m, Deep Learning, Image ClassificationAbstract
Malware represents an evolving cybersecurity threat that demands more effective detection methods. Conventional signature-based detection systems have limitations in identifying new variants, driving the development of deep learning-based approaches. This research implements and evaluates four variants of the YOLOv11 algorithm (n, s, m, l) for malware classification based on visual image representation. The dataset consists of 22,056 malware and benign images, divided into 70% training, 15% validation, and 15% testing across 8 classes (adware, backdoor, benign, downloader, spyware, trojan, virus, worm). Each model was trained for 100 epochs with batch size 32 using Google Colab with GPU support. Results demonstrate that all variants achieve high accuracy (97.8%-98.1%) with YOLOv11m as the best performer (98.1%). YOLOv11n offers optimal balance between accuracy (97.9%) and efficiency (1.5M parameters, 0.3 ms/img inference) ideal for real-time applications. This research surpasses previous methods such as K-NN (97.18%) and hybrid CNN (96.55%) with superior inference speed (0.3-0.9 ms/img vs tens to hundreds of ms/img), proving the effectiveness of YOLOv11 for fast, accurate, and scalable malware detection.
Downloads
References
J. Singh and K. K. Senapati, “Malware analysis and classification,” in Malware Analysis and Intrusion Detection in Cyber-Physical Systems, no. April, M. Kumar, Ed. IGI Global, 2023, pp. 42–63.
P. Maniriho, A. N. Mahmood, and M. J. M. Chowdhury, “A Survey of Recent Advances in Deep Learning Models for Detecting Malware in Desktop and Mobile Platforms,” ACM Comput. Surv., vol. 56, no. 6, pp. 145:1--145:41, Jan. 2024, doi: 10.1145/3638240.
M. Egele, T. Scholte, E. Kirda, and C. Kruegel, “A survey on automated dynamic malware-analysis techniques and tools,” ACM Comput. Surv., vol. 44, no. 2, pp. 1–49, 2012, doi: 10.1145/2089125.2089126.
L. Nataraj, S. Karthikeyan, G. Jacob, and B. S. Manjunath, “Malware images: visualization and automatic classification,” in Proceedings of the 8th International Symposium on Visualization for Cyber Security, Jul. 2011, pp. 1–7, doi: 10.1145/2016904.2016908.
A. Damodaran, F. Di Troia, C. A. Visaggio, T. H. Austin, and M. Stamp, “A comparison of static, dynamic, and hybrid analysis for malware detection,” J. Comput. Virol. Hacking Tech., vol. 13, no. 1, pp. 1–12, 2017, doi: 10.1007/s11416-015-0261-z.
A. Moser, C. Kruegel, and E. Kirda, “Limits of Static Analysis for Malware Detection,” in Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), 2007, pp. 421–430, doi: 10.1109/ACSAC.2007.21.
T. M. Mohammed, L. Nataraj, S. Chikkagoudar, S. Chandrasekaran, and B. S. Manjunath, “Malware detection using frequency domain-based image visualization and deep learning,” Proc. Annu. Hawaii Int. Conf. Syst. Sci., pp. 7132–7141, 2021, doi: 10.24251/hicss.2021.858.
S. Venkatraman, M. Alazab, and R. Vinayakumar, “A hybrid deep learning image-based analysis for effective malware detection,” J. Inf. Secur. Appl., vol. 47, pp. 377–389, 2019, doi: 10.1016/j.jisa.2019.06.006.
G. Plastiras, C. Kyrkou, and T. Theocharides, “Efficient convnet-based object detection for unmanned aerial vehicles by selective tile processing,” in ACM International Conference Proceeding Series, 2018, pp. 1–6, doi: 10.1145/3243394.3243692.
R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” pp. 1–9, 2024, doi: 10.48550/arXiv.2410.17725.
B. P. Gond, “Malware Benign Image Classification Dataset,” 2025. https://www.kaggle.com/datasets/bishwajitprasadgond/malware-benign-image-sample/data.
M. Alshoulie and A. Mehmood, “Deep Learning Approaches for Malware Detection: A Comprehensive Review of Techniques, Challenges, and Future Directions,” IEEE Access, vol. 13, pp. 118652–118677, 2025, doi: 10.1109/ACCESS.2025.3582875.
Y.-S. Chen and Y.-X. Chen, “The Application and Performance Comparison of Different Versions of YOLO Image Recognition Systems,” Eng. Proc., vol. 98, no. 1, pp. 1–7, 2025, doi: 10.3390/engproc2025098001.
Y. H. Chen, J. L. Chen, and R. F. Deng, “Similarity-Based Malware Classification Using Graph Neural Networks,” Appl. Sci., vol. 12, no. 21, 2022, doi: 10.3390/app122110837.
M. Kim, H. Cho, and J. H. Yi, “Large-Scale Analysis on Anti-Analysis Techniques in Real-World Malware,” IEEE Access, vol. 10, no. July, pp. 75802–75815, 2022, doi: 10.1109/ACCESS.2022.3190978.
I. Ahmad, Z. Wan, A. Ahmad, and S. S. Ullah, “A Hybrid Optimization Model for Efficient Detection and Classification of Malware in the Internet of Things,” Mathematics, vol. 12, no. 10, 2024, doi: 10.3390/math12101437.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Daniel Darren Richardo, Theophilus Wellem
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in IT-Explore: Jurnal Penerapan Teknologi Informasi dan Komunikasi is licensed under a Creative Commons Attribution 4.0 International License.
