Visible to the public Better Malware Ground Truth: Techniques for Weighting Anti-Virus Vendor LabelsConflict Detection Enabled

TitleBetter Malware Ground Truth: Techniques for Weighting Anti-Virus Vendor Labels
Publication TypeConference Paper
Year of Publication2015
AuthorsKantchelian, Alex, Tschantz, Michael Carl, Afroz, Sadia, Miller, Brad, Shankar, Vaishaal, Bachwani, Rekha, Joseph, Anthony D., Tygar, J. D.
Conference NameProceedings of the 8th ACM Workshop on Artificial Intelligence and Security
Conference LocationDenver, Colorado, USA
ISBN Number978-1-4503-3826-4
Keywordsaggregating labels, anti-virus vendors, expectation-maximization, Foundations, Hierarchical Coordination and Control, labeling problem, Resilient Systems, Science of decentralized security, science of security, SURE Project

We examine the problem of aggregating the results of multiple anti-virus (AV) vendors' detectors into a single authoritative ground-truth label for every binary. To do so, we adapt a well-known generative Bayesian model that postulates the existence of a hidden ground truth upon which the AV labels depend. We use training based on Expectation Maximization for this fully unsupervised technique. We evaluate our method using 279,327 distinct binaries from VirusTotal, each of which appeared for the rst time between January 2012 and June 2014.

Our evaluation shows that our statistical model is consistently more accurate at predicting the future-derived ground truth than all unweighted rules of the form \k out of n" AV detections. In addition, we evaluate the scenario where partial ground truth is available for model building. We train a logistic regression predictor on the partial label information. Our results show that as few as a 100 randomly selected training instances with ground truth are enough to achieve 80% true positive rate for 0.1% false positive rate. In comparison, the best unweighted threshold rule provides only 60% true positive rate at the same false positive rate.

Citation KeyKantchelian:2015:BMG:2808769.2808780