A Comprehensive Study on Malicious URL Detection: Leveraging Large-Scale Web Data for Accurate and Scalable Threat Identification

Abstract

The rapid growth of cyber-attacks launched through the internet, such as phishing, spreading of malware, and cyber-attacks involving hacking of websites, has added a sense of challenge in malicious URL detection. Conventional techniques that rely upon blacklists of malicious patterns lack efficient strategies for handling dynamically changing URLs. Keeping these limitations in mind, in the suggested research work, a novel approach has been introduced for malicious URL detection using techniques in deep learning and ensembling, wherein an efficient approach for classifying large-scale data is being proposed using Convolutional Neural Networks, Bidirectional Long Short-Term Memory, and XGBoost. The data on which experiments are carried out is a publicly available large-scale dataset that consists of more than 650,000 URLs, which can be classified as benign, phishing, defacement, and malware types. The model that is proposed in this research work is compared with other approaches using various baseline techniques such as logistic regression, SVM, XGBoost, and CNN. Performance parameters that are used are accuracy, precision, recall, F1 score, ROC curve, and confusion matrix. The experimental results have shown that the proposed model achieves an accuracy of 96%, compared to all the other models, and hence proves that simply by combining the concepts of deep sequential features and gradient boosting, a better model can be obtained that can give better results while detecting malicious URLs.