Visible to the public Zero-Day Attack Identification in Streaming Data Using Semantics and Spark

TitleZero-Day Attack Identification in Streaming Data Using Semantics and Spark
Publication TypeConference Paper
Year of Publication2017
AuthorsPallaprolu, S. C., Sankineni, R., Thevar, M., Karabatis, G., Wang, J.
Conference Name2017 IEEE International Congress on Big Data (BigData Congress)
ISBN Number978-1-5386-1996-4
Keywordsanomaly detection, Cognition, collaborative classification methods, Collaborative mining, composability, Computer hacking, data streaming, defense, dynamic semantic graph generation, feature extraction, feature selection, Flow Creation, graph theory, groupware, IDS, Intrusion Detection Systems, learning (artificial intelligence), Metrics, minimum redundancy maximum relevance feature selection algorithm, MRMR feature selection algorithm, parallel detection, pattern classification, pubcrawl, Resiliency, security of data, semantic learning, Semantic learning and reasoning, semantic link networks, semantic reasoning, Semantics, SLN, Spark Streaming, Spark streaming platform, Sparks, Training, Zero day attacks, zero-day attack identification

Intrusion Detection Systems (IDS) have been in existence for many years now, but they fall short in efficiently detecting zero-day attacks. This paper presents an organic combination of Semantic Link Networks (SLN) and dynamic semantic graph generation for the on the fly discovery of zero-day attacks using the Spark Streaming platform for parallel detection. In addition, a minimum redundancy maximum relevance (MRMR) feature selection algorithm is deployed to determine the most discriminating features of the dataset. Compared to previous studies on zero-day attack identification, the described method yields better results due to the semantic learning and reasoning on top of the training data and due to the use of collaborative classification methods. We also verified the scalability of our method in a distributed environment.

Citation Keypallaprolu_zero-day_2017