Visible to the public Hierarchical learning for automated malware classification

TitleHierarchical learning for automated malware classification
Publication TypeConference Paper
Year of Publication2017
AuthorsChakraborty, S., Stokes, J. W., Xiao, L., Zhou, D., Marinescu, M., Thomas, A.
Conference NameMILCOM 2017 - 2017 IEEE Military Communications Conference (MILCOM)
Date PublishedOct. 2017
ISBN Number978-1-5386-0595-0
Keywordsanti-virus companies, automated malware classification, binary error rate, classification models, commercial anti-virus products, Companies, computer security, computer viruses, corporate computers, family variant, feature extraction, formal specification, hierarchical labels, hierarchical learning algorithms, Hierarchical Machine Learning, hierarchical structure, home computers, Human Behavior, industrial-scale malware dataset, label hierarchy achieves, learning (artificial intelligence), machine learning algorithms, machine learning solutions, malicious file detection, malicious files automated classification, Malware, malware classification, malware label, Metrics, nonhierarchical classifier, pattern classification, privacy, pubcrawl, resilience, Resiliency, security domain, signature augmentation, Support vector machines, system monitoring, Training, Win32, Win64

Despite widespread use of commercial anti-virus products, the number of malicious files detected on home and corporate computers continues to increase at a significant rate. Recently, anti-virus companies have started investing in machine learning solutions to augment signatures manually designed by analysts. A malicious file's determination is often represented as a hierarchical structure consisting of a type (e.g. Worm, Backdoor), a platform (e.g. Win32, Win64), a family (e.g. Rbot, Rugrat) and a family variant (e.g. A, B). While there has been substantial research in automated malware classification, the aforementioned hierarchical structure, which can provide additional information to the classification models, has been ignored. In this paper, we propose the novel idea and study the performance of employing hierarchical learning algorithms for automated classification of malicious files. To the best of our knowledge, this is the first research effort which incorporates the hierarchical structure of the malware label in its automated classification and in the security domain, in general. It is important to note that our method does not require any additional effort by analysts because they typically assign these hierarchical labels today. Our empirical results on a real world, industrial-scale malware dataset of 3.6 million files demonstrate that incorporation of the label hierarchy achieves a significant reduction of 33.1% in the binary error rate as compared to a non-hierarchical classifier which is traditionally used in such problems.

Citation Keychakraborty_hierarchical_2017