Visible to the public A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction

TitleA Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction
Publication TypeConference Paper
Year of Publication2018
AuthorsSultana, K. Z., Williams, B. J., Bosu, A.
Conference Name2018 25th Asia-Pacific Software Engineering Conference (APSEC)
Date Published Dec. 2018
ISBN Number978-1-7281-1970-0
KeywordsComplexity theory, Internet, Java, learning (artificial intelligence), Measurement, method-level software metrics, method-level traceable patterns, Metrics, metrics testing, nano-pattern, nanopatterns, Object oriented modeling, Predictive models, program testing, pubcrawl, security, security of data, Software, software metrics, software quality, software security, vulnerabilities, Vulnerability, vulnerability prediction models, vulnerable code, vulnerable methods, web services

Context: Software security is an imperative aspect of software quality. Early detection of vulnerable code during development can better ensure the security of the codebase and minimize testing efforts. Although traditional software metrics are used for early detection of vulnerabilities, they do not clearly address the granularity level of the issue to precisely pinpoint vulnerabilities. The goal of this study is to employ method-level traceable patterns (nano-patterns) in vulnerability prediction and empirically compare their performance with traditional software metrics. The concept of nano-patterns is similar to design patterns, but these constructs can be automatically recognized and extracted from source code. If nano-patterns can better predict vulnerable methods compared to software metrics, they can be used in developing vulnerability prediction models with better accuracy. Aims: This study explores the performance of method-level patterns in vulnerability prediction. We also compare them with method-level software metrics. Method: We studied vulnerabilities reported for two major releases of Apache Tomcat (6 and 7), Apache CXF, and two stand-alone Java web applications. We used three machine learning techniques to predict vulnerabilities using nano-patterns as features. We applied the same techniques using method-level software metrics as features and compared their performance with nano-patterns. Results: We found that nano-patterns show lower false negative rates for classifying vulnerable methods (for Tomcat 6, 21% vs 34.7%) and therefore, have higher recall in predicting vulnerable code than the software metrics used. On the other hand, software metrics show higher precision than nano-patterns (79.4% vs 76.6%). Conclusion: In summary, we suggest developers use nano-patterns as features for vulnerability prediction to augment existing approaches as these code constructs outperform standard metrics in terms of prediction recall.

Citation Keysultana_comparison_2018