Visible to the public Biblio

Filters: Keyword is benchmark  [Clear All Filters]
2021-04-27
Aigner, A., Khelil, A..  2020.  A Benchmark of Security Metrics in Cyber-Physical Systems. 2020 IEEE International Conference on Sensing, Communication and Networking (SECON Workshops). :1—6.

The usage of connected devices and their role within our daily- and business life gains more and more impact. In addition, various derivations of Cyber-Physical Systems (CPS) reach new business fields, like smart healthcare or Industry 4.0. Although these systems do bring many advantages for users by extending the overall functionality of existing systems, they come with several challenges, especially for system engineers and architects. One key challenge consists in achieving a sufficiently high level of security within the CPS environment, as sensitive data or safety-critical functions are often integral parts of CPS. Being system of systems (SoS), CPS complexity, unpredictability and heterogeneity complicate analyzing the overall level of security, as well as providing a way to detect ongoing attacks. Usually, security metrics and frameworks provide an effective tool to measure the level of security of a given component or system. Although several comprehensive surveys exist, an assessment of the effectiveness of the existing solutions for CPS environments is insufficiently investigated in literature. In this work, we address this gap by benchmarking a carefully selected variety of existing security metrics in terms of their usability for CPS. Accordingly, we pinpoint critical CPS challenges and qualitatively assess the effectiveness of the existing metrics for CPS systems.

2020-08-14
Gu, Zuxing, Wu, Jiecheng, Liu, Jiaxiang, Zhou, Min, Gu, Ming.  2019.  An Empirical Study on API-Misuse Bugs in Open-Source C Programs. 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC). 1:11—20.
Today, large and complex software is developed with integrated components using application programming interfaces (APIs). Correct usage of APIs in practice presents a challenge due to implicit constraints, such as call conditions or call orders. API misuse, i.e., violation of these constraints, is a well-known source of bugs, some of which can cause serious security vulnerabilities. Although researchers have developed many API-misuse detectors over the last two decades, recent studies show that API misuses are still prevalent. In this paper, we provide a comprehensive empirical study on API-misuse bugs in open-source C programs. To understand the nature of API misuses in practice, we analyze 830 API-misuse bugs from six popular programs across different domains. For all the studied bugs, we summarize their root causes, fix patterns and usage statistics. Furthermore, to understand the capabilities and limitations of state-of-the-art static analysis detectors for API-misuse detection, we develop APIMU4C, a dataset of API-misuse bugs in C code based on our empirical study results, and evaluate three widely-used detectors on it qualitatively and quantitatively. We share all the findings and present possible directions towards more powerful API-misuse detectors.
Mitra, Joydeep, Ranganath, Venkatesh-Prasad, Narkar, Aditya.  2019.  BenchPress: Analyzing Android App Vulnerability Benchmark Suites. 2019 34th IEEE/ACM International Conference on Automated Software Engineering Workshop (ASEW). :13—18.
In recent years, various benchmark suites have been developed to evaluate the efficacy of Android security analysis tools. Tool developers often choose such suites based on the availability and popularity of suites and not on their characteristics and relevance due to the lack of information about them. In this context, based on a recent effort, we empirically evaluated four Android-specific benchmark suites: DroidBench, Ghera, ICCBench, and UBCBench. For each benchmark suite, we identified the APIs used by the suite that were discussed on Stack Overflow in the context of Android app development and measured the usage of these APIs in a sample of 227K real-world apps (coverage). We also identified security-related APIs used in real-world apps but not in any of the above benchmark suites to assess the opportunities to extend benchmark suites (gaps).
2019-06-17
Yang, Lishan, Cherkasova, Ludmila, Badgujar, Rajeev, Blancaflor, Jack, Konde, Rahul, Mills, Jason, Smirni, Evgenia.  2018.  Evaluating Scalability and Performance of a Security Management Solution in Large Virtualized Environments. Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering. :168–175.
Virtualized infrastructure is a key capability of modern enterprise data centers and cloud computing, enabling a more agile and dynamic IT infrastructure with fast IT provisioning, simplified, automated management, and flexible resource allocation to handle a broad set of workloads. However, at the same time, virtualization introduces new challenges, since securing virtual servers is more difficult than physical machines. HyTrust Inc. has developed an innovative security solution, called HyTrust Cloud Control (HTCC), to mitigate risks associated with virtualization and cloud technologies. HTCC is a virtual appliance deployed as a transparent proxy in front of a VMware-based virtualized environment. Since HTCC serves as a gateway to a customer virtualized environment, it is important to carefully assess its performance and scalability as well as provide its accurate resource sizing. In this work, we introduce a novel approach for accomplishing this goal. First, we describe a special framework, based on a nested virtualization technique, which enables the creation and deployment of a large scale virtualized environment (with 30,000 VMs) using a limited number of physical servers (4 servers in our experiments). Second, we introduce a design and implementation of a novel, extensible benchmark, called HT-vmbench, that allows to mimic the session-based activities of different system administrators and users in virtualized environments. The benchmark is implemented using VMware Web Service SDK. By executing HT-vmbench in the emulated large-scale virtualized environments, we can support an efficient performance assessment of management and security solutions (such as HTCC), their overhead, and provide capacity planning rules and resource sizing recommendations.
2019-03-15
Ye, J., Yang, Y., Gong, Y., Hu, Y., Li, X..  2018.  Grey Zone in Pre-Silicon Hardware Trojan Detection. 2018 IEEE International Test Conference in Asia (ITC-Asia). :79-84.

Pre-Silicon hardware Trojan detection has been studied for years. The most popular benchmark circuits are from the Trust-Hub. Their common feature is that the probability of activating hardware Trojans is very low. This leads to a series of machine learning based hardware Trojan detection methods which try to find the nets with low signal probability of 0 or 1. On the other hand, it is considered that, if the probability of activating hardware Trojans is high, these hardware Trojans can be easily found through behaviour simulations or during functional test. This paper explores the "grey zone" between these two opposite scenarios: if the activation probability of a hardware Trojan is not low enough for machine learning to detect it and is not high enough for behaviour simulation or functional test to find it, it can escape from detection. Experiments show the existence of such hardware Trojans, and this paper suggests a new set of hardware Trojan benchmark circuits for future study.

2018-12-03
Bernin, Arne, Müller, Larissa, Ghose, Sobin, von Luck, Kai, Grecos, Christos, Wang, Qi, Vogt, Florian.  2017.  Towards More Robust Automatic Facial Expression Recognition in Smart Environments. Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments. :37–44.

In this paper, we provide insights towards achieving more robust automatic facial expression recognition in smart environments based on our benchmark with three labeled facial expression databases. These databases are selected to test for desktop, 3D and smart environment application scenarios. This work is meant to provide a neutral comparison and guidelines for developers and researchers interested to integrate facial emotion recognition technologies in their applications, understand its limitations and adaptation as well as enhancement strategies. We also introduce and compare three different metrics for finding the primary expression in a time window of a displayed emotion. In addition, we outline facial emotion recognition limitations and enhancements for smart environments and non-frontal setups. By providing our comparison and enhancements we hope to build a bridge from affective computing research and solution providers to application developers that like to enhance new applications by including emotion based user modeling.

2018-06-07
Mlinarić, Danijel, Mornar, Vedran.  2017.  Dynamic Software Updating in Java: Comparing Concepts and Resource Demands. Companion to the First International Conference on the Art, Science and Engineering of Programming. :12:1–12:6.

Dynamic software updating (DSU) is an extremely useful feature to be used during software evolution. It can be used to reduce down-time costs, for security enhancements, profiling and testing new functionalities. There are many studies and solutions on dynamic software updating regarding diverse problems introduced by the topic, but there is a lack of research which compares various approaches concerning supported changes and demands on resources. In this paper, we are comparing currently available concepts for Java programming language that deal with dynamically applied changes and measuring the impact of those changes on computer resource demands.

2018-05-09
Lokananta, F., Hartono, D., Tang, C. M..  2017.  A Scalable and Reconfigurable Verification and Benchmark Environment for Network on Chip Architecture. 2017 4th International Conference on New Media Studies (CONMEDIA). :6–10.

To reduce the complex communication problem that arise as the number of on-chip component increases, the use of Network-on-Chip (NoC) as interconnection architectures have become more promising to solve complex on-chip communication problems. However, providing a suitable test base to measure and verify functionality of any NoC is a compulsory. Universal Verification Methodology (UVM) is introduced as a standardized and reusable methodology for verifying integrated circuit design. In this research, a scalable and reconfigurable verification and benchmark environment for NoC is proposed.

2017-05-18
Amani, Sven, Nadi, Sarah, Nguyen, Hoan A., Nguyen, Tien N., Mezini, Mira.  2016.  MUBench: A Benchmark for API-misuse Detectors. Proceedings of the 13th International Conference on Mining Software Repositories. :464–467.

Over the last few years, researchers proposed a multitude of automated bug-detection approaches that mine a class of bugs that we call API misuses. Evaluations on a variety of software products show both the omnipresence of such misuses and the ability of the approaches to detect them. This work presents MuBench, a dataset of 89 API misuses that we collected from 33 real-world projects and a survey. With the dataset we empirically analyze the prevalence of API misuses compared to other types of bugs, finding that they are rare, but almost always cause crashes. Furthermore, we discuss how to use it to benchmark and compare API-misuse detectors.