Visible to the public Biblio

Filters: Keyword is Predictive Metrics  [Clear All Filters]
Hunag, C., Yang, C., Weng, C., Chen, Y., Wang, S..  2019.  Secure Protocol for Identity-based Provable Data Possession in Cloud Storage. 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS). :327–331.
Remote data possession is becoming an increasingly important issue in cloud storage. It enables users to verify if their outsourced data have remained intact while in cloud storage. The existing remote data audit (RDA) protocols were designed with the public key infrastructure (PKI) system. However, this incurs considerable costs when users need to frequently access data from the cloud service provider with PKI. This study proposes a protocol, called identity-based RDA (ID-RDA) that addresses this problem without the need for users’ certificates. This study outperforms existing RDA protocols in computation and communication.
Eugster, P., Marson, G. A., Poettering, B..  2018.  A Cryptographic Look at Multi-party Channels. 2018 IEEE 31st Computer Security Foundations Symposium (CSF). :31–45.
Cryptographic channels aim to enable authenticated and confidential communication over the Internet. The general understanding seems to be that providing security in the sense of authenticated encryption for every (unidirectional) point-to-point link suffices to achieve this goal. As recently shown (in FSE17/ToSC17), however, the security properties of the unidirectional links do not extend, in general, to the bidirectional channel as a whole. Intuitively, the reason for this is that the increased interaction in bidirectional communication can be exploited by an adversary. The same applies, a fortiori, in a multi-party setting where several users operate concurrently and the communication develops in more directions. In the cryptographic literature, however, the targeted goals for group communication in terms of channel security are still unexplored. Applying the methodology of provable security, we fill this gap by defining exact (game-based) authenticity and confidentiality goals for broadcast communication, and showing how to achieve them. Importantly, our security notions also account for the causal dependencies between exchanged messages, thus naturally extending the bidirectional case where causal relationships are automatically captured by preserving the sending order. On the constructive side we propose a modular and yet efficient protocol that, assuming only point-to-point links between users, leverages (non-cryptographic) broadcast and standard cryptographic primitives to a full-fledged broadcast channel that provably meets the security notions we put forth.
Aydin, Y., Ozkaynak, F..  2019.  A Provable Secure Image Encryption Schema Based on Fractional Order Chaotic Systems. 2019 23rd International Conference Electronics. :1–5.
In the literature, many chaotic systems have been used in the design of image encryption algorithms. In this study, an application of fractional order chaotic systems is investigated. The aim of the study is to improve the disadvantageous aspects of existing methods based on discrete and continuous time chaotic systems by utilizing the features of fractional order chaotic systems. The most important advantage of the study compared to the literature is that the proposed encryption algorithm is designed with a provable security approach. Analyses results have been shown that the proposed method can be used successfully in many information security applications.
Moon, J., Lee, Y., Yang, H., Song, T., Won, D..  2018.  Cryptanalysis of a privacy-preserving and provable user authentication scheme for wireless sensor networks based on Internet of Things security. 2018 International Conference on Information Networking (ICOIN). :432–437.
User authentication in wireless sensor networks is more complex than normal networks due to sensor network characteristics such as unmanned operation, limited resources, and unreliable communication. For this reason, various authentication protocols have been presented to provide secure and efficient communication. In 2017, Wu et al. presented a provable and privacy-preserving user authentication protocol for wireless sensor networks. Unfortunately, we found that Wu et al.'s protocol was still vulnerable against user impersonation attack, and had a problem in the password change phase. We show how an attacker can impersonate an other user and why the password change phase is ineffective.
Ammar, Mahmoud, Daniels, Wilfried, Crispo, Bruno, Hughes, Danny.  2018.  SPEED: Secure Provable Erasure for Class-1 IoT Devices. Proceedings of the Eighth ACM Conference on Data and Application Security and Privacy. :111–118.
The Internet of Things (IoT) consists of embedded devices that sense and manage our environment in a growing range of applications. Large-scale IoT systems such as smart cities require significant investment in both equipment and personnel. To maximize return on investment, IoT platforms should support multiple third-party applications and adaptation of infrastructure over time. Realizing the vision of shared IoT platforms demands strong security guarantees. That is particularly challenging considering the limited capability and resource constraints of many IoT devices. In this paper, we present SPEED, an approach to secure erasure with verifiability in IoT. Secure erasure is a fundamental property when it comes to share an IoT platform with other users which guarantees the cleanness of a device's memory at the beginning of the application deployment as well as at the time of releasing the underlying IoT device. SPEED relies on two security primitives: memory isolation and distance bounding protocol. We evaluate the performance of SPEED by implementing it on a simple bare-metal IoT device belongs to Class-1. Our evaluation results show a limited overhead in terms of memory footprint, time, and energy consumption.
Sahin, Cetin, Kuczenski, Brandon, Egecioglu, Omer, El Abbadi, Amr.  2018.  Privacy-Preserving Certification of Sustainability Metrics. Proceedings of the Eighth ACM Conference on Data and Application Security and Privacy. :53–63.
Companies are often motivated to evaluate their environmental sustainability, and to make public pronouncements about their performance with respect to quantitative sustainability metrics. Public trust in these declarations is enhanced if the claims are certified by a recognized authority. Because accurate evaluations of environmental impacts require detailed information about industrial processes throughout a supply chain, protecting the privacy of input data in sustainability assessment is of paramount importance. We introduce a new paradigm, called privacy-preserving certification, that enables the computation of sustainability indicators in a privacy-preserving manner, allowing firms to be classified based on their individual performance without revealing sensitive information to the certifier, other parties, or the public. In this work, we describe different variants of the certification problem, highlight the necessary security requirements, and propose a provably-secure novel framework that performs the certification operations under the management of an authorized, yet untrusted, party without compromising confidential information.
Duan, Li, Li, Yong, Liao, Lijun.  2018.  Flexible Certificate Revocation List for Efficient Authentication in IoT. Proceedings of the 8th International Conference on the Internet of Things. :7:1–7:8.
When relying on public key infrastructure (PKI) for authentication, whether a party can be trusted primarily depends on its certificate status. Bob's certificate status can be retrieved by Alice through her interaction with Certificate Authority (CA) in the PKI. More specifically, Alice can download Certificate Revocation List (CRL) and then check whether the serial number of the Bob's certificate appears in this list. If not found, Alice knows that Bob can be trusted. Once downloaded, a CRL can be used offline for arbitrary many times till it expires, which saves the bandwidth to an extreme. However, if the number of revoked certificates becomes too large, the size of the CRL will exceed the RAM of Alice's device. This conflict between bandwidth and RAM consumption becomes even more challenging for the Internet-of-Things (IoT), since the IoT end-devices is usually constrained by both factors. To solve this problem in PKI-based authentication in IoT, we proposed two novel lightweight CRL protocols with maximum flexibility tailored for constrained IoT end-devices. The first one is based on generalized Merkle hash tree and the second is based on Bloom filter. We also provided quantitative theorems for CRL parameter configuration, which help strike perfect balance among bandwidth, RAM usage and security in various practical IoT scenarios. Furthermore, we thoroughly evaluated the proposed CRL protocols and exhibited their outstanding efficiency in terms of RAM and bandwidth consumption. In addition, our formal treatment of the security of a CRL protocol can also be of independent interest.
Kesidis, G., Shan, Y., Fleck, D., Stavrou, A., Konstantopoulos, T..  2018.  An adversarial coupon-collector model of asynchronous moving-target defense against botnet reconnaissance*. 2018 13th International Conference on Malicious and Unwanted Software (MALWARE). :61–67.
We consider a moving-target defense of a proxied multiserver tenant of the cloud where the proxies dynamically change to defeat reconnaissance activity by a botnet planning a DDoS attack targeting the tenant. Unlike the system of [4] where all proxies change simultaneously at a fixed rate, we consider a more “responsive” system where the proxies may change more rapidly and selectively based on the current session request intensity, which is expected to be abnormally large during active reconnaissance. In this paper, we study a tractable “adversarial” coupon-collector model wherein proxies change after a random period of time from the latest request, i.e., asynchronously. In addition to determining the stationary mean number of proxies discovered by the attacker, we study the age of a proxy (coupon type) when it has been identified (requested) by the botnet. This gives us the rate at which proxies change (cost to the defender) when the nominal client request load is relatively negligible.
Narantuya, J., Yoon, S., Lim, H., Cho, J., Kim, D. S., Moore, T., Nelson, F..  2019.  SDN-Based IP Shuffling Moving Target Defense with Multiple SDN Controllers. 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks – Supplemental Volume (DSN-S). :15–16.
Conventional SDN-based MTD techniques have been mainly developed with a single SDN controller which exposes a single point of failure as well as raises a scalability issue for large-scale networks in achieving both security and performance. The use of multiple SDN controllers has been proposed to ensure both performance and security of SDN-based MTD systems for large-scale networks; however, the effect of using multiple SDN controllers has not been investigated in the state-of-the-art research. In this paper, we propose the SDN based MTD architecture using multiple SDN controllers and validate their security effect (i.e., attack success probability) by implementing an IP shuffling MTD in a testbed using ONOS SDN controllers.
Zhang, Z., Yu, Q., Njilla, L., Kamhoua, C..  2018.  FPGA-oriented moving target defense against security threats from malicious FPGA tools. 2018 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). :163–166.
The imbalance relationship between FPGA hardware/software providers and FPGA users challenges the assurance of secure design on FPGAs. Existing efforts on FPGA security primarily focus on reverse engineering the downloaded FPGA configuration, retrieving the authentication code or crypto key stored on the embedded memory in FPGAs, and countermeasures for the security threats above. In this work, we investigate new security threats from malicious FPGA tools, and identify stealthy attacks that could occur during FPGA deployment. To address those attacks, we exploit the principles of moving target defense (MTD) and propose a FPGA-oriented MTD (FOMTD) method. Our method is composed of three defense lines, which are formed by an improved user constraint file, random selection of design replicas, and runtime submodule assembling, respectively. The FPGA emulation results show that the proposed FOMTD method reduces the hardware Trojan hit rate by 60% over the baseline, at the cost of 10.76% more power consumption.
Macwan, S., Lung, C..  2019.  Investigation of Moving Target Defense Technique to Prevent Poisoning Attacks in SDN. 2019 IEEE World Congress on Services (SERVICES). 2642-939X:178–183.
The motivation behind Software-Defined Networking (SDN) is to allow services and network capabilities to be managed through a central control point. Moving Target Defense (MTD) introduces a constantly changing environment in order to delay or prevent attacks on a system. For the effective use of MTD, SDN can be used to help confuse the attacker from gathering legitimate information about the network. This paper investigates how SDN can be used for some network based MTD techniques and evaluate the benefits of integrating techniques in SDN and MTD. In the experiment, network assets are kept hidden from inside and outside attackers. Furthermore, the SDN controller is programed to perform IP mutation to keep changing real IP addresses of the underlying hosts by assigning each host a virtual IP address at a configured mutation rate to prevent attackers from stealing the real IP addresses or using fake IP addresses. The paper demonstrates experimental evaluation of the MTD technique using the Ryu controller and mininet. The results show that the MTD technique can be easily integrated into the SDN environment to use virtual IP addresses for hosts to reduce the chance of poisoning attacks.
Wang, S., Zhou, Y., Guo, R., Du, J., Du, J..  2018.  A Novel Route Randomization Approach for Moving Target Defense. 2018 IEEE 18th International Conference on Communication Technology (ICCT). :11–15.
Route randomization is an important research focus for moving target defense which seeks to proactively and dynamically change the forwarding routes in the network. In this paper, the difficulties of implementing route randomization in traditional networks are analyzed. To solve these difficulties and achieve effective route randomization, a novel route randomization approach is proposed, which is implemented by adding a mapping layer between routers' physical interfaces and their corresponding logical addresses. The design ideas and the details of proposed approach are presented. The effectiveness and performance of proposed approach are verified and evaluated by corresponding experiments.
Almohaimeed, A., Asaduzzaman, A..  2019.  A Novel Moving Target Defense Technique to Secure Communication Links in Software-Defined Networks. 2019 Fifth Conference on Mobile and Secure Services (MobiSecServ). :1–4.
Software-defined networking (SDN) is a recently developed approach to computer networking that brings a centralized orientation to network control, thereby improving network architecture and management. However, as with any communication environment that involves message transmission among users, SDN is confronted by the ongoing challenge of protecting user privacy. In this “Work in Progress (WIP)” research, we propose an SDN security model that applies the moving target defense (MTD) technique to protect communication links from sensitive data leakages. MTD is a security solution aimed at increasing complexity and uncertainty for attackers by concealing sensitive information that may serve as a gateway from which to launch different types of attacks. The proposed MTD-based security model is intended to protect user identities contained in transmitted messages in a way that prevents network intruders from identifying the real identities of senders and receivers. According to the results from preliminary experiments, the proposed MTD model has potential to protect the identities contained in transmitted messages within communication links. This work will be extended to protect sensitive data if an attacker gets access to the network device.
Zhou, X., Lu, Y., Wang, Y., Yan, X..  2018.  Overview on Moving Target Network Defense. 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). :821–827.
Moving Target Defense (MTD) is a research hotspot in the field of network security. Moving Target Network Defense (MTND) is the implementation of MTD at network level. Numerous related works have been proposed in the field of MTND. In this paper, we focus on the scope and area of MTND, systematically present the recent representative progress from four aspects, including IP address and port mutation, route mutation, fingerprint mutation and multiple mutation, and put forward the future development directions. Several new perspectives and elucidations on MTND are rendered.
Chowdhary, Ankur, Alshamrani, Adel, Huang, Dijiang, Liang, Hongbin.  2018.  MTD Analysis and Evaluation Framework in Software Defined Network (MASON). Proceedings of the 2018 ACM International Workshop on Security in Software Defined Networks & Network Function Virtualization. :43–48.
Security issues in a Software Defined Network (SDN) environment like system vulnerabilities and intrusion attempts can pose a security risk for multi-tenant network managed by SDN. In this research work, Moving target defense (MTD)technique based on shuffle strategy - port hopping has been employed to increase the difficulty for the attacker trying to exploit the cloud network. Our research workMASON, considers the problem of multi-stage attacks in a network managed using SDN. SDN controller can be used to dynamically reconfigure the network and render attacker»s knowledge in multi-stage attacks redundant. We have used a threat score based on vulnerability information and intrusion attempts to identify Virtual Machines (VMs) in systems with high-security risk and implement MTD countermeasures port hopping to assess threat score reduction in a cloud network.
Fraunholz, Daniel, Krohmer, Daniel, Duque Anton, Simon, Schotten, Hans Dieter.  2018.  Catch Me If You Can: Dynamic Concealment of Network Entities. Proceedings of the 5th ACM Workshop on Moving Target Defense. :31–39.
In this paper, a framework for Moving Target Defense is introduced. This framework bases on three pillars: network address mutation, communication stack randomization and the dynamic deployment of decoys. The network address mutation is based on the concept of domain generation algorithms, where different features are included to fulfill the system requirements. Those requirements are time dependency, unpredictability and determinism. Communication stack randomization is applied additionally to increase the complexity of reconnaissance activity. By employing communication stack randomization, previously fingerprinted systems do not only differ in the network address but also in their communication pattern behavior. And finally, decoys are integrated into the proposed framework to detect attackers that have breached the perimeter. Furthermore, attacker's resources can be bound by interacting with the decoy systems. Additionally, the framework can be extended with more advanced Moving Target Defense methods such as obscuring port numbers of services.
Mulamba, Dieudonne, Amarnath, Athith, Bezawada, Bruhadeshwar, Ray, Indrajit.  2018.  A Secure Hash Commitment Approach for Moving Target Defense of Security-critical Services. Proceedings of the 5th ACM Workshop on Moving Target Defense. :59–68.
Protection of security-critical services, such as access-control reference monitors, is an important requirement in the modern era of distributed systems and services. The threat arises from hosting the service on a single server for a lengthy period of time, which allows the attacker to periodically enumerate the vulnerabilities of the service with respect to the server's configuration and launch targeted attacks on the service. In our work, we design and implement an efficient solution based on the moving "target" defense strategy, to protect security-critical services against such active adversaries. Specifically, we focus on implementing our solution for protecting the reference monitor service that enforces access control for users requesting access to sensitive resources. The key intuition of our approach is to increase the level of difficulty faced by the attacker to compromise a service by periodically moving the security-critical service among a group of heterogeneous servers. For this approach to be practically feasible, the movement of the service should be efficient and random, i.e., the attacker should not have a-priori information about the choice of the next server hosting the service. Towards this, we describe an efficient Byzantine fault-tolerant leader election protocol that achieves the desired security and performance objectives. We built a prototype implementation that moves the access control service randomly among a group of fifty servers within a time range of 250-440 ms. We show that our approach tolerates Byzantine behavior of servers, which ensures that a server under adversarial control has no additional advantage of being selected as the next active server.
Zhao, Guangsheng, Xiong, Xinli, Wu, Huaying.  2018.  A Model for Analyzing the Effectiveness of Moving Target Defense. Proceedings of the 8th International Conference on Communication and Network Security. :17–21.
Moving target defense(MTD) is a typical proactive cyber defense technology, which not only increases the difficulty of the attacker, but also reduces the damage caused by successful attacks. A number of studies have assessed the defensive effectiveness of MTD, but only focus on increasing the difficulty of attacks. No studies have been conducted to assess the impact of successful attacks on the network. In this paper, we propose a probability model that evaluates the impact of MTD against subsequent stages of complete attack process. The model quantify the probability distribution of the number of compromised hosts. The results of simulation show that MTD can reduce the number of compromised hosts, and attackers cannot control all hosts.
Connell, Warren, Pham, Luan Huy, Philip, Samuel.  2018.  Analysis of Concurrent Moving Target Defenses. Proceedings of the 5th ACM Workshop on Moving Target Defense. :21–30.
While Moving Target Defenses (MTDs) have been increasingly recognized as a promising direction for cyber security, quantifying the effects of MTDs remains mostly an open problem. Each MTD has its own set of advantages and disadvantages. No single MTD provides an effective defense against the entire range of possible threats. One of the challenges facing MTD quantification efforts is predicting the cumulative effect of implementing multiple MTDs. We present a scenario where two MTDs are deployed in an experimental testbed created to model a realistic use case. This is followed by a probabilistic analysis of the effectiveness of both MTDs against a multi-step attack, along with the MTDs' impact on availability to legitimate users. Our work is essential to providing decision makers with the knowledge to make informed choices regarding cyber defense.
Liu, Jed, Corbett-Davies, Joe, Ferraiuolo, Andrew, Ivanov, Alexander, Luo, Mulong, Suh, G. Edward, Myers, Andrew C., Campbell, Mark.  2018.  Secure Autonomous Cyber-Physical Systems Through Verifiable Information Flow Control. Proceedings of the 2018 Workshop on Cyber-Physical Systems Security and PrivaCy. :48–59.

Modern cyber-physical systems are complex networked computing systems that electronically control physical systems. Autonomous road vehicles are an important and increasingly ubiquitous instance. Unfortunately, their increasing complexity often leads to security vulnerabilities. Network connectivity exposes these vulnerable systems to remote software attacks that can result in real-world physical damage, including vehicle crashes and loss of control authority. We introduce an integrated architecture to provide provable security and safety assurance for cyber-physical systems by ensuring that safety-critical operations and control cannot be unintentionally affected by potentially malicious parts of the system. Fine-grained information flow control is used to design both hardware and software, determining how low-integrity information can affect high-integrity control decisions. This security assurance is used to improve end-to-end security across the entire cyber-physical system. We demonstrate this integrated approach by developing a mobile robotic testbed modeling a self-driving system and testing it with a malicious attack.

Champagne, Samuel, Makanju, Tokunbo, Yao, Chengchao, Zincir-Heywood, Nur, Heywood, Malcolm.  2018.  A Genetic Algorithm for Dynamic Controller Placement in Software Defined Networking. Proceedings of the Genetic and Evolutionary Computation Conference Companion. :1632–1639.

The Software Defined Networking paradigm has enabled dynamic configuration and control of large networks. Although the division of the control and data planes on networks has lead to dynamic reconfigurability of large networks, finding the minimal and optimal set of controllers that can adapt to the changes in the network has proven to be a challenging problem. Recent research tends to favor small solution sets with a focus on either propagation latency or controller load distribution, and struggles to find large balanced solution sets. In this paper, we propose a multi-objective genetic algorithm based approach to the controller placement problem that minimizes inter-controller latency, load distribution and the number of controllers with fitness sharing. We demonstrate that the proposed approach provides diverse and adaptive solutions to real network architectures such as the United States backbone and Japanese backbone networks. We further discuss the relevance and application of a diversity focused genetic algorithm for a moving target defense security model.

Fraunholz, Daniel, Reti, Daniel, Duque Anton, Simon, Schotten, Hans Dieter.  2018.  Cloxy: A Context-aware Deception-as-a-Service Reverse Proxy for Web Services. Proceedings of the 5th ACM Workshop on Moving Target Defense. :40–47.

Legacy software, outdated applications and fast changing technologies pose a serious threat to information security. Several domains, such as long-life industrial control systems and Internet of Things devices, suffer from it. In many cases, system updates and new acquisitions are not an option. In this paper, a framework that combines a reverse proxy with various deception-based defense mechanisms is presented. It is designed to autonomously provide deception methods to web applications. Context-awareness and minimal configuration overhead make it perfectly suited to work as a service. The framework is built modularly to provide flexibility and adaptability to the application use case. It is evaluated with common web-based applications such as content management systems and several frequent attack vectors against them. Furthermore, the security and performance implications of the additional security layer are quantified and discussed. It is found that, given sound implementation, no further attack vectors are introduced to the web application. The performance of the prototypical framework increases the delay of communication with the underlying web application. This delay is within tolerable boundaries and can be further reduced by a more efficient implementation.

Xiong, Xinli, Zhao, Guangsheng, Wang, Xian.  2018.  A System Attack Surface Based MTD Effectiveness and Cost Quantification Framework. Proceedings of the 2Nd International Conference on Cryptography, Security and Privacy. :175-179.

Moving Target Defense (MTD) is a game-changing method to thwart adversaries and reverses the imbalance situation in network countermeasures. Introducing Attack Surface (AS) into MTD security assessment brings productive concepts to qualitative and quantitative analysis. The quantification of MTD effectiveness and cost (E&C) has been under researched, using simulation models and emulation testbeds, to give accurate and reliable results for MTD technologies. However, the lack of system-view evaluation impedes MTD to move toward large-scale applications. In this paper, a System Attack Surface Based Quantification Framework (SASQF) is proposed to establish a system-view based framework for further research in Attack Surface and MTD E&C quantification. And a simulated model based on SASQF is developed to provide illustrations and software simulation methods. A typical C/S scenario and Cyber Kill Chain (CKC) attacks are presented in case study and several simulated results are given. From the simulated results, IP mutation frequency is the key to increase consumptions of adversaries, while the IP mutation pool is not the principal factor to thwart adversaries in reconnaissance and delivery of CKC steps. For system user operational cost, IP mutation frequency influence legitimate connections in relative values under ideal link state without delay, packet lose and jitter. The simulated model based on SASQF also provides a basic method to find the optimal IP mutation frequency through simulations.

Potteiger, Bradley, Zhang, Zhenkai, Koutsoukos, Xenofon.  2018.  Integrated Instruction Set Randomization and Control Reconfiguration for Securing Cyber-physical Systems. Proceedings of the 5th Annual Symposium and Bootcamp on Hot Topics in the Science of Security. :5:1–5:10.

Cyber-Physical Systems (CPS) have been increasingly subject to cyber-attacks including code injection attacks. Zero day attacks further exasperate the threat landscape by requiring a shift to defense in depth approaches. With the tightly coupled nature of cyber components with the physical domain, these attacks have the potential to cause significant damage if safety-critical applications such as automobiles are compromised. Moving target defense techniques such as instruction set randomization (ISR) have been commonly proposed to address these types of attacks. However, under current implementations an attack can result in system crashing which is unacceptable in CPS. As such, CPS necessitate proper control reconfiguration mechanisms to prevent a loss of availability in system operation. This paper addresses the problem of maintaining system and security properties of a CPS under attack by integrating ISR, detection, and recovery capabilities that ensure safe, reliable, and predictable system operation. Specifically, we consider the problem of detecting code injection attacks and reconfiguring the controller in real-time. The developed framework is demonstrated with an autonomous vehicle case study.

Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J. P., Humbert, M., Juels, A., Lin, H..  2017.  FairTest: Discovering Unwarranted Associations in Data-Driven Applications. 2017 IEEE European Symposium on Security and Privacy (EuroS P). :401–416.

In a world where traditional notions of privacy are increasingly challenged by the myriad companies that collect and analyze our data, it is important that decision-making entities are held accountable for unfair treatments arising from irresponsible data usage. Unfortunately, a lack of appropriate methodologies and tools means that even identifying unfair or discriminatory effects can be a challenge in practice. We introduce the unwarranted associations (UA) framework, a principled methodology for the discovery of unfair, discriminatory, or offensive user treatment in data-driven applications. The UA framework unifies and rationalizes a number of prior attempts at formalizing algorithmic fairness. It uniquely combines multiple investigative primitives and fairness metrics with broad applicability, granular exploration of unfair treatment in user subgroups, and incorporation of natural notions of utility that may account for observed disparities. We instantiate the UA framework in FairTest, the first comprehensive tool that helps developers check data-driven applications for unfair user treatment. It enables scalable and statistically rigorous investigation of associations between application outcomes (such as prices or premiums) and sensitive user attributes (such as race or gender). Furthermore, FairTest provides debugging capabilities that let programmers rule out potential confounders for observed unfair effects. We report on use of FairTest to investigate and in some cases address disparate impact, offensive labeling, and uneven rates of algorithmic error in four data-driven applications. As examples, our results reveal subtle biases against older populations in the distribution of error in a predictive health application and offensive racial labeling in an image tagger.