A DDoS Attack Detection System Based on Spark Framework

Dezhi Han1, Kun Bi1, Han Liu1 and Jianxin Jia1

  1. College of Information Engineering, Shanghai Maritime University
    Shanghai 201306, China
    fdzhan, kunbig@shmtu.edu.cn, fjmakg23,onlyonemang@163.com


There are many problems in traditional Distributed Denial of Service (DDoS) attack detection such as low accuracy, low detection speed and so on, which is not suitable for the real time detecting and processing of DDoS attacks in big data environment. This paper proposed a novel DDoS attack detection system based on Spark framework including 3 main algorithms. Based on information entropy, the first one can effectively warn all kinds of DDoS attacks in advance according to the information entropy change of data stream source IP address and destination IP address; With the help of designed dynamic sampling K-Means algorithm, this new detection system improves the attack detection accuracy effectively; Through running dynamic sampling K-Means parallelization algorithm, which can quickly and effectively detect a variety of DDoS attacks in big data environment. The experiment results show that this system can not only early warn DDoS attacks effectively, but also can detect all kinds of DDoS attacks in real time, with low false rate.

Key words

Distributed Denial of Service (DDoS), Early Warn, Attack Detection, Spark framework, K-Means Algorithm

Digital Object Identifier (DOI)


Publication information

Volume 14, Issue 3 (September 2017)
Advances in Information Technology, Distributed and Model Driven Systems
Year of Publication: 2017
ISSN: 2406-1018 (Online)
Publisher: ComSIS Consortium

Full text

DownloadAvailable in PDF
Portable Document Format

How to cite

Han, D., Bi, K., Liu, H., Jia, J.: A DDoS Attack Detection System Based on Spark Framework. Computer Science and Information Systems, Vol. 14, No. 3, 769–788. (2017), https://doi.org/10.2298/CSIS161217028H