Visible to the public A Graph-Based Model for Malicious Software Detection Exploiting Domination Relations Between System-Call Groups

TitleA Graph-Based Model for Malicious Software Detection Exploiting Domination Relations Between System-Call Groups
Publication TypeConference Paper
Year of Publication2018
AuthorsMpanti, Anna, Nikolopoulos, Stavros D., Polenakis, Iosif
Conference NameProceedings of the 19th International Conference on Computer Systems and Technologies
PublisherACM
ISBN Number978-1-4503-6425-6
KeywordsAlgorithms, detection, graph theory, graphs, Human Behavior, Malware, malware analysis, Metrics, privacy, pubcrawl, Resiliency, security, Systems
Abstract

In this paper, we propose a graph-based algorithmic technique for malware detection, utilizing the System-call Dependency Graphs (ScDG) obtained through taint analysis traces. We leverage the grouping of system-calls into system-call groups with respect to their functionality to merge disjoint vertices of ScDG graphs, transforming them to Group Relation Graphs (GrG); note that, the GrG graphs represent malware's behavior being hence more resilient to probable mutations of its structure. More precisely, we extend the use of GrG graphs by mapping their vertices on the plane utilizing the degrees and the vertex-weights of a specific underlying graph of the GrG graph as to compute domination relations. Furthermore, we investigate how the activity of each system-call group could be utilized in order to distinguish graph-representations of malware and benign software. The domination relations among the vertices of GrG graphs result to a new graph representation that we call Coverage Graph of the GrG graph. Finally, we evaluate the potentials of our detection model using graph similarity between Coverage Graphs of known malicious and benign software samples of various types.

URLhttps://dl.acm.org/citation.cfm?doid=3274005.3274028
DOI10.1145/3274005.3274028
Citation Keympanti_graph-based_2018