Visible to the public Establishing a Baseline for Measuring Advancement in the Science of Security - an Analysis of the 2015 IEEE Security & Privacy ProceedingsConflict Detection Enabled

TitleEstablishing a Baseline for Measuring Advancement in the Science of Security - an Analysis of the 2015 IEEE Security & Privacy Proceedings
Publication TypeConference Paper
Year of Publication2016
AuthorsCarver, J., Burcham, M., Kocak, S., Bener, A., Felderer, M., Gander, M., King, J., Markkula, J., Oivo, M., Sauerwein, C., Williams, L.
Conference Name2016 Symposium and Bootcamp on the Science of Security (HotSoS)
Date Published04/2016
Conference LocationPittsburgh, PA
KeywordsA Human Information-Processing Analysis of Online Deception Detection, Apr'16, Attack Surface and Defense-in-Depth Metrics, Automated Synthesis of Resilient Architectures, Formal Specification and Analysis of Security-Critical Norms and Policies, Leveraging the Effects of Cognitive Function on Input Device Analytics to Improve Security, NCSU, NCSU PROJECTS:(select all that apply), Privacy Incidents Database, Redundancy for Network Intrusion Prevention Systems (NIPS), Resilience Requirements, Design, and Testing, Scientific Understanding of Policy Complexity, Smart Isolation in Large-Scale Production Computing Infrastructures, Systematization of Knowledge from Intrusion Detection Models, Understanding the Effects of Norms and Policies on the Robustness, Liveness, and Resilience of Systems, Vulnerability and Resilience Prediction Models, Warning of Phishing Attacks: Supporting Human Information Processing, Identifying Phishing Deception Indicators & Reducing Vuln.

To help establish a more scientific basis for security science, which will enable the development of fundamental theories and move the field from being primarily reactive to primarily proactive, it is important for research results to be reported in a scientifically rigorous manner. Such reporting will allow for the standard pillars of science, namely replication, meta-analysis, and theory building. In this paper we aim to establish a baseline of the state of scientific work in security through the analysis of indicators of scientific research as reported in the papers from the 2015 IEEE Symposium on Security and Privacy. To conduct this analysis, we developed a series of rubrics to determine the completeness of the papers relative to the type of evaluation used (e.g. case study, experiment, proof). Our findings showed that while papers are generally easy to read, they often do not explicitly document some key information like the research objectives, the process for choosing the cases to include in the studies, and the threats to validity. We hope that this initial analysis will serve as a baseline against which we can measure the advancement of the science of security.

Citation Keynode-25885
Refereed DesignationRefereed