Network Intrusion Detection Using Transformer Models and Natural Language Processing for Enhanced Web Application Attack Detection

Wowon Priatna; Irwan  Sembiring; Adi Setiawan; Iwan Setyawan

doi:10.23887/janapati.v13i3.82462

Authors

Wowon Priatna Universitas Bhayangakara Jakarta Raya
Irwan Sembiring Universitas Kristen Satya Wacana
Adi Setiawan Universitas Kristen Satya Wacana
Iwan Setyawan Universitas Kristen Satya Wacana

DOI:

https://doi.org/10.23887/janapati.v13i3.82462

Keywords:

NLP, intrusion detection, transformer, machine learning, web application attack

Abstract

The increasing frequency and complexity of web application attacks necessitate more advanced detection methods. This research explores integrating Transformer models and Natural Language Processing (NLP) techniques to enhance network intrusion detection systems (NIDS). Traditional NIDS often rely on predefined signatures and rules, limiting their effectiveness against new attacks. By leveraging the Transformer's ability to capture long-term dependencies and the contextual richness of NLP, this study aims to develop a more adaptive and intelligent intrusion detection framework. Utilizing the CSIC 2010 dataset, comprehensive preprocessing steps such as tokenization, stemming, lemmatization, and normalization were applied. Techniques like Word2Vec, BERT, and TF-IDF were used for text representation, followed by the application of the Transformer architecture. Performance evaluation using accuracy, precision, recall, F1 score, and AUC demonstrated the superiority of the Transformer-NLP model over traditional machine learning methods. Statistical validation through Friedman and T-tests confirmed the model's robustness and practical significance. Despite promising results, limitations include the dataset's scope, computational complexity, and the need for further research to generalize the model to other types of network attacks. This study indicates significant improvements in detecting complex web application attacks, reducing false positives, and enhancing overall security, making it a viable solution for addressing increasingly sophisticated cybersecurity threats

References

A. A. Bouramdane, “Cyberattacks in Smart Grids: Challenges and Solving the Multi-Criteria Decision-Making for Cybersecurity Options, Including Ones That Incorporate Artificial Intelligence, Using an Analytical Hierarchy Process,” J. Cybersecurity Priv., vol. 3, no. 4, pp. 662–705, 2023, doi: 10.3390/jcp3040031.

J. A. Dharma and Rino, “Network Attack Detection Using Intrusion Detection System Utilizing Snort Based on Telegram,” bit-Tech, vol. 6, no. 2, pp. 118–126, 2023, doi: 10.32877/bt.v6i2.943.

O. J. Falana, I. O. Ebo, C. O. Tinubu, O. A. Adejimi, and A. Ntuk, “Detection of Cross-Site Scripting Attacks using Dynamic Analysis and Fuzzy Inference System,” 2020 Int. Conf. Math. Comput. Eng. Comput. Sci. ICMCECS 2020, 2020, doi: 10.1109/ICMCECS47690.2020.240871.

P. Dini, A. Elhanashi, A. Begni, S. Saponara, Q. Zheng, and K. Gasmi, “Overview on Intrusion Detection Systems Design Exploiting Machine Learning for Networking Cybersecurity,” Appl. Sci., vol. 13, no. 13, 2023, doi: 10.3390/app13137507.

N. Agarwal and S. Z. Hussain, “A Closer Look at Intrusion Detection System for Web Applications,” Secur. Commun. Networks, vol. 2018, 2018, doi: 10.1155/2018/9601357.

Y. J. Park and J. C. Park, “Web Application Intrusion Detection System for input validation attack,” Proc. - 3rd Int. Conf. Converg. Hybrid Inf. Technol. ICCIT 2008, vol. 2, pp. 498–504, 2008, doi: 10.1109/ICCIT.2008.338.

S. Sasipriya, L. R. Madhan Kumar, R. Raghuram Krishnan, and K. Naveen Kumar, “Intrusion Detection System in Web Applications (IDSWA),” Proc. - 5th Int. Conf. Intell. Comput. Control Syst. ICICCS 2021, no. Iciccs, pp. 311–314, 2021, doi: 10.1109/ICICCS51141.2021.9432086.

M. Verkerken, L. D’hooge, T. Wauters, B. Volckaert, and F. De Turck, “Towards Model Generalization for Intrusion Detection: Unsupervised Machine Learning Techniques,” J. Netw. Syst. Manag., vol. 30, no. 1, pp. 1–25, 2022, doi: 10.1007/s10922-021-09615-7.

L. Ashiku and C. Dagli, “Network Intrusion Detection System using Deep Learning,” Procedia Comput. Sci., vol. 185, no. June, pp. 239–247, 2021, doi: 10.1016/j.procs.2021.05.025.

R. Sudiyarno, A. Setyanto, and E. T. Luthfi, “Peningkatan Performa Pendeteksian Anomali Menggunakan Ensemble Learning dan Feature Selection Anomaly Detection Performance Improvement Using Ensemble Learning and Feature Selection,” Citec J., vol. 7, no. 1, pp. 1–9, 2020.

S. Das, M. Ashrafuzzaman, F. T. Sheldon, and S. Shiva, “Network Intrusion Detection using Natural Language Processing and Ensemble Machine Learning,” 2020 IEEE Symp. Ser. Comput. Intell. SSCI 2020, no. Ml, pp. 829–835, 2020, doi: 10.1109/SSCI47803.2020.9308268.

R. Sujatha, A. Teja, P. Naveen, and J. M. Chatterjee, “Web Application for Traffic Monitoring and Guidance,” vol. 10, no. 4, pp. 1–14, 2020, doi: 10.33168/JSMS.2020.0403.

J. R. Tadhani, V. Vekariya, V. Sorathiya, S. Alshathri, and W. El Shafai, “Securing web applications against XSS and SQLi attacks using a novel deep learning approach,” Sci. Rep., pp. 1–17, 2024, doi: 10.1038/s41598-023-48845-4.

T. Sowmya and M. A. E. A, “Measurement : Sensors A comprehensive review of AI based intrusion detection system,” Meas. Sensors, vol. 28, no. May, p. 100827, 2023, doi: 10.1016/j.measen.2023.100827.

J. Campino, “Unleashing the transformers : NLP models detect AI writing in education,” J. Comput. Educ., no. 0123456789, 2024, doi: 10.1007/s40692-024-00325-y.

N. Patwardhan, S. Marrone, and C. Sansone, “Transformers in the Real World : A Survey on NLP Applications,” 2023.

Z. Long, H. Yan, G. Shen, X. Zhang, H. He, and L. Cheng, “A Transformer-based network intrusion detection approach for cloud security,” J. Cloud Comput., vol. 13, no. 1, 2024, doi: 10.1186/s13677-023-00574-9.

Y. Liu and L. Wu, “Intrusion Detection Model Based on Improved Transformer,” Appl. Sci., vol. 13, no. 10, 2023, doi: 10.3390/app13106251.

J. Kim, H. Kang, and P. Kang, “Time-series anomaly detection with stacked Transformer representations and 1D convolutional network,” Eng. Appl. Artif. Intell., vol. 120, no. November 2022, p. 105964, 2023, doi: 10.1016/j.engappai.2023.105964.

N. Montes, G. Betarte, R. Martínez, and A. Pardo, “Web Application Attacks Detection Using Deep Learning,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12702 LNCS, pp. 227–236, 2021, doi: 10.1007/978-3-030-93420-0_22.

A. D. Y. SURYADI, “Pengembangan Intrusion Detection System (IDS) Berbasis Machine Learning,” vol. 13, no. 3, pp. 189–195, 2022, [Online]. Available: https://repository.mercubuana.ac.id/63488/.

A. Nurdin, B. Anggo Seno Aji, A. Bustamin, and Z. Abidin, “Perbandingan Kinerja Word Embedding Word2Vec, Glove, Dan Fasttext Pada Klasifikasi Teks,” J. Tekno Kompak, vol. 14, no. 2, p. 74, 2020, doi: 10.33365/jtk.v14i2.732.

S. R. Choi and M. Lee, “Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review,” Biology (Basel)., vol. 12, no. 7, 2023, doi: 10.3390/biology12071033.

H. Salih Abdullah and A. Mohsin Abdulazeez, “Detection of SQL Injection Attacks Based on Supervised Machine Learning Algorithms: A Review,” Int. J. Informatics, Inf. Syst. Comput. Eng., vol. 5, no. 2, pp. 152–165, 2024, doi: 10.34010/injiiscom.v5i2.12731.

H. Wang and W. Li, “DDosTC: A transformer-based network attack detection hybrid mechanism in SDN,” Sensors, vol. 21, no. 15, 2021, doi: 10.3390/s21155047.

Z. Gao, Y. Shi, and S. Li, “Self-attention and long-range relationship capture network for underwater object detection,” J. King Saud Univ. - Comput. Inf. Sci., vol. 36, no. 2, p. 101971, 2024, doi: 10.1016/j.jksuci.2024.101971.

H. Kheddar, “Transformers and Large Language Models for Efficient Intrusion Detection Systems : A Comprehensive Survey,” pp. 1–34.

H. Zhang and M. O. Shafiq, “Survey of transformers and towards ensemble learning using transformers for natural language processing,” J. Big Data, 2024, doi: 10.1186/s40537-023-00842-0.

D. E. Cahyani and I. Patasik, “Performance comparison of TF-IDF and Word2Vec models for emotion text classification,” vol. 10, no. 5, pp. 2780–2788, 2021, doi: 10.11591/eei.v10i5.3157.

G. Zachos, I. Essop, G. Mantas, K. Porfyrakis, J. C. Ribeiro, and J. Rodriguez, “An Anomaly-Based Intrusion Detection System for Internet of Medical Things Networks,” Electronics, no. June, pp. 1–25, 2021.

A. Aldallal, “Toward Efficient Intrusion Detection System Using Hybrid Deep Learning Approach,” Sysmmetry, 2022.

A. Chandra, L. Tünnermann, T. Löfstedt, and R. Gratz, “Transformer-based deep learning for predicting protein properties in the life sciences,” Elife, vol. 12, pp. 1–25, 2023, doi: 10.7554/eLife.82819.

T. Lin, Y. Wang, X. Liu, and X. Qiu, “A survey of transformers,” AI Open, vol. 3, no. October, pp. 111–132, 2022, doi: 10.1016/j.aiopen.2022.10.001.

R. Cao, J. Wang, M. Mao, G. Liu, and C. Jiang, “Feature-wise attention based boosting ensemble method for fraud detection,” Eng. Appl. Artif. Intell., vol. 126, no. PC, p. 106975, 2023, doi: 10.1016/j.engappai.2023.106975.

T. S. Lestari, I. Ismaniah, and W. Priatna, “Particle Swarm Optimization for Optimizing Public Service Satisfaction Level Classification,” J. Nas. Pendidik. Tek. Inform., vol. 13, no. 1, pp. 147–155, 2024, doi: 10.23887/janapati.v13i1.69612.

J. Liu and Y. Xu, “T-Friedman Test: A New Statistical Test for Multiple Comparison with an Adjustable Conservativeness Measure,” Int. J. Comput. Intell. Syst., vol. 15, no. 1, pp. 1–19, 2022, doi: 10.1007/s44196-022-00083-8.

W. Priatna, H. Dwi Purnomo, A. Iriani, I. Sembiring, and T. Wellem, “Optimizing Multilayer Perceptron with Cost-Sensitive Learning for Addressing Class Imbalance in Credit Card Fraud Detection,” Resti, vol. 8, no. 4, pp. 19–25, 2024.