Visible to the public Biblio

Filters: Keyword is black box  [Clear All Filters]
2021-03-09
Rahmati, A., Moosavi-Dezfooli, S.-M., Frossard, P., Dai, H..  2020.  GeoDA: A Geometric Framework for Black-Box Adversarial Attacks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). :8443–8452.
Adversarial examples are known as carefully perturbed images fooling image classifiers. We propose a geometric framework to generate adversarial examples in one of the most challenging black-box settings where the adversary can only generate a small number of queries, each of them returning the top-1 label of the classifier. Our framework is based on the observation that the decision boundary of deep networks usually has a small mean curvature in the vicinity of data samples. We propose an effective iterative algorithm to generate query-efficient black-box perturbations with small p norms which is confirmed via experimental evaluations on state-of-the-art natural image classifiers. Moreover, for p=2, we theoretically show that our algorithm actually converges to the minimal perturbation when the curvature of the decision boundary is bounded. We also obtain the optimal distribution of the queries over the iterations of the algorithm. Finally, experimental results confirm that our principled black-box attack algorithm performs better than state-of-the-art algorithms as it generates smaller perturbations with a reduced number of queries.
Herrera, A. E. Hinojosa, Walshaw, C., Bailey, C..  2020.  Improving Black Box Classification Model Veracity for Electronics Anomaly Detection. 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA). :1092–1097.
Data driven classification models are useful to assess quality of manufactured electronics. Because decisions are taken based on the models, their veracity is relevant, covering aspects such as accuracy, transparency and clarity. The proposed BB-Stepwise algorithm aims to improve the classification model transparency and accuracy of black box models. K-Nearest Neighbours (KNN) is a black box model which is easy to implement and has achieved good classification performance in different applications. In this paper KNN-Stepwise is illustrated for fault detection of electronics devices. The results achieved shows that the proposed algorithm was able to improve the accuracy, veracity and transparency of KNN models and achieve higher transparency and clarity, and at least similar accuracy than when using Decision Tree models.
Guibene, K., Ayaida, M., Khoukhi, L., MESSAI, N..  2020.  Black-box System Identification of CPS Protected by a Watermark-based Detector. 2020 IEEE 45th Conference on Local Computer Networks (LCN). :341–344.
The implication of Cyber-Physical Systems (CPS) in critical infrastructures (e.g., smart grids, water distribution networks, etc.) has introduced new security issues and vulnerabilities to those systems. In this paper, we demonstrate that black-box system identification using Support Vector Regression (SVR) can be used efficiently to build a model of a given industrial system even when this system is protected with a watermark-based detector. First, we briefly describe the Tennessee Eastman Process used in this study. Then, we present the principal of detection scheme and the theory behind SVR. Finally, we design an efficient black-box SVR algorithm for the Tennessee Eastman Process. Extensive simulations prove the efficiency of our proposed algorithm.
Rojas-Dueñas, G., Riba, J., Kahalerras, K., Moreno-Eguilaz, M., Kadechkar, A., Gomez-Pau, A..  2020.  Black-Box Modelling of a DC-DC Buck Converter Based on a Recurrent Neural Network. 2020 IEEE International Conference on Industrial Technology (ICIT). :456–461.
Artificial neural networks allow the identification of black-box models. This paper proposes a method aimed at replicating the static and dynamic behavior of a DC-DC power converter based on a recurrent nonlinear autoregressive exogenous neural network. The method proposed in this work applies an algorithm that trains a neural network based on the inputs and outputs (currents and voltages) of a Buck converter. The approach is validated by means of simulated data of a realistic nonsynchronous Buck converter model programmed in Simulink and by means of experimental results. The predictions made by the neural network are compared to the actual outputs of the system, to determine the accuracy of the method, thus validating the proposed approach. Both simulation and experimental results show the feasibility and accuracy of the proposed black-box approach.
Mashhadi, M. J., Hemmati, H..  2020.  Hybrid Deep Neural Networks to Infer State Models of Black-Box Systems. 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). :299–311.
Inferring behavior model of a running software system is quite useful for several automated software engineering tasks, such as program comprehension, anomaly detection, and testing. Most existing dynamic model inference techniques are white-box, i.e., they require source code to be instrumented to get run-time traces. However, in many systems, instrumenting the entire source code is not possible (e.g., when using black-box third-party libraries) or might be very costly. Unfortunately, most black-box techniques that detect states over time are either univariate, or make assumptions on the data distribution, or have limited power for learning over a long period of past behavior. To overcome the above issues, in this paper, we propose a hybrid deep neural network that accepts as input a set of time series, one per input/output signal of the system, and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns, over time. We have applied our approach on a real UAV auto-pilot solution from our industry partner with half a million lines of C code. We ran 888 random recent system-level test cases and inferred states, over time. Our comparison with several traditional time series change point detection techniques showed that our approach improves their performance by up to 102%, in terms of finding state change points, measured by F1 score. We also showed that our state classification algorithm provides on average 90.45% F1 score, which improves traditional classification algorithms by up to 17%.
Cui, W., Li, X., Huang, J., Wang, W., Wang, S., Chen, J..  2020.  Substitute Model Generation for Black-Box Adversarial Attack Based on Knowledge Distillation. 2020 IEEE International Conference on Image Processing (ICIP). :648–652.
Although deep convolutional neural network (CNN) performs well in many computer vision tasks, its classification mechanism is very vulnerable when it is exposed to the perturbation of adversarial attacks. In this paper, we proposed a new algorithm to generate the substitute model of black-box CNN models by using knowledge distillation. The proposed algorithm distills multiple CNN teacher models to a compact student model as the substitution of other black-box CNN models to be attacked. The black-box adversarial samples can be consequently generated on this substitute model by using various white-box attacking methods. According to our experiments on ResNet18 and DenseNet121, our algorithm boosts the attacking success rate (ASR) by 20% by training the substitute model based on knowledge distillation.
MATSUNAGA, Y., AOKI, N., DOBASHI, Y., KOJIMA, T..  2020.  A Black Box Modeling Technique for Distortion Stomp Boxes Using LSTM Neural Networks. 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). :653–656.
This paper describes an experimental result of modeling stomp boxes of the distortion effect based on a machine learning approach. Our proposed technique models a distortion stomp box as a neural network consisting of LSTM layers. In this approach, the neural network is employed for learning the nonlinear behavior of the distortion stomp boxes. All the parameters for replicating the distortion sound are estimated through its training process using the input and output signals obtained from some commercial stomp boxes. The experimental result indicates that the proposed technique may have a certain appropriateness to replicate the distortion sound by using the well-trained neural networks.
2021-03-04
Kalin, J., Ciolino, M., Noever, D., Dozier, G..  2020.  Black Box to White Box: Discover Model Characteristics Based on Strategic Probing. 2020 Third International Conference on Artificial Intelligence for Industries (AI4I). :60—63.

In Machine Learning, White Box Adversarial Attacks rely on knowing underlying knowledge about the model attributes. This works focuses on discovering to distrinct pieces of model information: the underlying architecture and primary training dataset. With the process in this paper, a structured set of input probes and the output of the model become the training data for a deep classifier. Two subdomains in Machine Learning are explored - image based classifiers and text transformers with GPT-2. With image classification, the focus is on exploring commonly deployed architectures and datasets available in popular public libraries. Using a single transformer architecture with multiple levels of parameters, text generation is explored by fine tuning off different datasets. Each dataset explored in image and text are distinguishable from one another. Diversity in text transformer outputs implies further research is needed to successfully classify architecture attribution in text domain.

Crescenzo, G. D., Bahler, L., McIntosh, A..  2020.  Encrypted-Input Program Obfuscation: Simultaneous Security Against White-Box and Black-Box Attacks. 2020 IEEE Conference on Communications and Network Security (CNS). :1—9.

We consider the problem of protecting cloud services from simultaneous white-box and black-box attacks. Recent research in cryptographic program obfuscation considers the problem of protecting the confidentiality of programs and any secrets in them. In this model, a provable program obfuscation solution makes white-box attacks to the program not more useful than black-box attacks. Motivated by very recent results showing successful black-box attacks to machine learning programs run by cloud servers, we propose and study the approach of augmenting the program obfuscation solution model so to achieve, in at least some class of application scenarios, program confidentiality in the presence of both white-box and black-box attacks.We propose and formally define encrypted-input program obfuscation, where a key is shared between the entity obfuscating the program and the entity encrypting the program's inputs. We believe this model might be of interest in practical scenarios where cloud programs operate over encrypted data received by associated sensors (e.g., Internet of Things, Smart Grid).Under standard intractability assumptions, we show various results that are not known in the traditional cryptographic program obfuscation model; most notably: Yao's garbled circuit technique implies encrypted-input program obfuscation hiding all gates of an arbitrary polynomial circuit; and very efficient encrypted-input program obfuscation for range membership programs and a class of machine learning programs (i.e., decision trees). The performance of the latter solutions has only a small constant overhead over the equivalent unobfuscated program.

2021-03-01
Houzé, É, Diaconescu, A., Dessalles, J.-L., Mengay, D., Schumann, M..  2020.  A Decentralized Approach to Explanatory Artificial Intelligence for Autonomic Systems. 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C). :115–120.
While Explanatory AI (XAI) is attracting increasing interest from academic research, most AI-based solutions still rely on black box methods. This is unsuitable for certain domains, such as smart homes, where transparency is key to gaining user trust and solution adoption. Moreover, smart homes are challenging environments for XAI, as they are decentralized systems that undergo runtime changes. We aim to develop an XAI solution for addressing problems that an autonomic management system either could not resolve or resolved in a surprising manner. This implies situations where the current state of affairs is not what the user expected, hence requiring an explanation. The objective is to solve the apparent conflict between expectation and observation through understandable logical steps, thus generating an argumentative dialogue. While focusing on the smart home domain, our approach is intended to be generic and transferable to other cyber-physical systems offering similar challenges. This position paper focuses on proposing a decentralized algorithm, called D-CAN, and its corresponding generic decentralized architecture. This approach is particularly suited for SISSY systems, as it enables XAI functions to be extended and updated when devices join and leave the managed system dynamically. We illustrate our proposal via several representative case studies from the smart home domain.
Kuppa, A., Le-Khac, N.-A..  2020.  Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security. 2020 International Joint Conference on Neural Networks (IJCNN). :1–8.

Cybersecurity community is slowly leveraging Machine Learning (ML) to combat ever evolving threats. One of the biggest drivers for successful adoption of these models is how well domain experts and users are able to understand and trust their functionality. As these black-box models are being employed to make important predictions, the demand for transparency and explainability is increasing from the stakeholders.Explanations supporting the output of ML models are crucial in cyber security, where experts require far more information from the model than a simple binary output for their analysis. Recent approaches in the literature have focused on three different areas: (a) creating and improving explainability methods which help users better understand the internal workings of ML models and their outputs; (b) attacks on interpreters in white box setting; (c) defining the exact properties and metrics of the explanations generated by models. However, they have not covered, the security properties and threat models relevant to cybersecurity domain, and attacks on explainable models in black box settings.In this paper, we bridge this gap by proposing a taxonomy for Explainable Artificial Intelligence (XAI) methods, covering various security properties and threat models relevant to cyber security domain. We design a novel black box attack for analyzing the consistency, correctness and confidence security properties of gradient based XAI methods. We validate our proposed system on 3 security-relevant data-sets and models, and demonstrate that the method achieves attacker's goal of misleading both the classifier and explanation report and, only explainability method without affecting the classifier output. Our evaluation of the proposed approach shows promising results and can help in designing secure and robust XAI methods.

2020-09-28
Simos, Dimitris E., Garn, Bernhard, Zivanovic, Jovan, Leithner, Manuel.  2019.  Practical Combinatorial Testing for XSS Detection using Locally Optimized Attack Models. 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). :122–130.
In this paper, we present a combinatorial testing methodology for automated black-box security testing of complex web applications. The focus of our work is the identification of Cross-site Scripting (XSS) vulnerabilities. We introduce a new modelling scheme for test case generation of XSS attack vectors consisting of locally optimized attack models. The modelling approach takes into account the response and behavior of the web application and is particularly efficient when used in conjunction with combinatorial testing. In addition to the modelling scheme, we present a research prototype of a security testing tool called XSSInjector, which executes attack vectors generated from our methodology against web applications. The tool also employs a newly developed test oracle for detecting XSS which allow us to precisely identify whether injected JavaScript is actually executed and thus eliminate false positives. Our testing methodology is sufficiently generic to be applied to any web application that returns HTML code. We describe the foundations of our approach and validate it via an extensive case study using a verification framework and real world web applications. In particular, we have found several new critical vulnerabilities in popular forum software, library management systems and gallery packages.
2020-09-04
Shi, Yang, Zhang, Qing, Liang, Jingwen, He, Zongjian, Fan, Hongfei.  2019.  Obfuscatable Anonymous Authentication Scheme for Mobile Crowd Sensing. IEEE Systems Journal. 13:2918—2929.
Mobile crowd sensing (MCS) is a rapidly developing technique for information collection from the users of mobile devices. This technique deals with participants' personal information such as their identities and locations, thus raising significant security and privacy concerns. Accordingly, anonymous authentication schemes have been widely considered for preserving participants' privacy in MCS. However, mobile devices are easy to lose and vulnerable to device capture attacks, which enables an attacker to extract the private authentication key of a mobile application and to further invade the user's privacy by linking sensed data with the user's identity. To address this issue, we have devised a special anonymous authentication scheme where the authentication request algorithm can be obfuscated into an unintelligible form and thus the authentication key is not explicitly used. This scheme not only achieves authenticity and unlinkability for participants, but also resists impersonation, replay, denial-of-service, man-in-the-middle, collusion, and insider attacks. The scheme's obfuscation algorithm is the first obfuscator for anonymous authentication, and it satisfies the average-case secure virtual black-box property. The scheme also supports batch verification of authentication requests for improving efficiency. Performance evaluations on a workstation and smart phones have indicated that our scheme works efficiently on various devices.
Teng, Jikai, Ma, Hongyang.  2019.  Dynamic asymmetric group key agreement protocol with traitor traceability. IET Information Security. 13:703—710.
In asymmetric group key agreement (ASGKA) protocols, a group of users establish a common encryption key which is publicly accessible and compute pairwise different decryption keys. It is left as an open problem to design an ASGKA protocol with traitor traceability in Eurocrypt 2009. A one-round dynamic authenticated ASGKA protocol with public traitor traceability is proposed in this study. It provides a black-box tracing algorithm. Ind-CPA security with key compromise impersonation resilience (KCIR) and forward secrecy of ASGKA protocols is formally defined. The proposed protocol is proved to be Ind-CPA secure with KCIR and forward secrecy under D k-HDHE assumption. It is also proved that the proposed protocol resists collusion attack. In Setup algorithm and Join algorithm, one communication round is required. In Leave algorithm, no message is required to be transmitted. The proposed protocol adopts O(log N)-way asymmetric multilinear map to make the size of public key and the size of ciphertext both achieve O(logN), where N is the number of potential group members. This is the first ASGKA protocol with public traitor traceability which is more efficient than trivial construction of ASGKA protocols.
Li, Chengqing, Feng, Bingbing, Li, Shujun, Kurths, Jüergen, Chen, Guanrong.  2019.  Dynamic Analysis of Digital Chaotic Maps via State-Mapping Networks. IEEE Transactions on Circuits and Systems I: Regular Papers. 66:2322—2335.
Chaotic dynamics is widely used to design pseudo-random number generators and for other applications, such as secure communications and encryption. This paper aims to study the dynamics of the discrete-time chaotic maps in the digital (i.e., finite-precision) domain. Differing from the traditional approaches treating a digital chaotic map as a black box with different explanations according to the test results of the output, the dynamical properties of such chaotic maps are first explored with a fixed-point arithmetic, using the Logistic map and the Tent map as two representative examples, from a new perspective with the corresponding state-mapping networks (SMNs). In an SMN, every possible value in the digital domain is considered as a node and the mapping relationship between any pair of nodes is a directed edge. The scale-free properties of the Logistic map's SMN are proved. The analytic results are further extended to the scenario of floating-point arithmetic and for other chaotic maps. Understanding the network structure of a chaotic map's SMN in digital computers can facilitate counteracting the undesirable degeneration of chaotic dynamics in finite-precision domains, also helping to classify and improve the randomness of pseudo-random number sequences generated by iterating the chaotic maps.
Li, Ge, Iyer, Vishnuvardhan, Orshansky, Michael.  2019.  Securing AES against Localized EM Attacks through Spatial Randomization of Dataflow. 2019 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). :191—197.
A localized electromagnetic (EM) attack is a potent threat to security of embedded cryptographic implementations. The attack utilizes high resolution EM probes to localize and exploit information leakage in sub-circuits of a system, providing information not available in traditional EM and power attacks. In this paper, we propose a countermeasure based on randomizing the assignment of sensitive data to parallel datapath components in a high-performance implementation of AES. In contrast to a conventional design where each state register byte is routed to a fixed S-box, a permutation network, controlled by a transient random value, creates a dynamic random mapping between the state registers and the set of S-boxes. This randomization results in a significant reduction of exploitable leakage.We demonstrate the countermeasure's effectiveness under two attack scenarios: a more powerful attack that assumes a fully controlled access to an attacked implementation for building a priori EM-profiles, and a generic attack based on the black-box model. Spatial randomization leads to a 150× increase of the minimum traces to disclosure (MTD) for the profiled attack and a 3.25× increase of MTD for the black-box model attack.
Qin, Baodong, Zheng, Dong.  2019.  Generic Approach to Outsource the Decryption of Attribute-Based Encryption in Cloud Computing. IEEE Access. 7:42331—42342.
The notion of attribute-based encryption with outsourced decryption (OD-ABE) was proposed by Green, Hohenberger, and Waters. In OD-ABE, the ABE ciphertext is converted to a partially-decrypted ciphertext that has a shorter bit length and a faster decryption time than that of the ABE ciphertext. In particular, the transformation can be performed by a powerful third party with a public transformation key. In this paper, we propose a generic approach for constructing ABE with outsourced decryption from standard ABE, as long as the later satisfies some additional properties. Its security can be reduced to the underlying standard ABE in the selective security model by a black-box way. To avoid the drawback of selective security in practice, we further propose a modified decryption outsourcing mode so that our generic construction can be adapted to satisfying adaptive security. This partially solves the open problem of constructing an OD-ABE scheme, and its adaptive security can be reduced to the underlying ABE scheme in a black-box way. Then, we present some concrete constructions that not only encompass existing ABE outsourcing schemes of Green et al., but also result in new selectively/adaptively-secure OD-ABE schemes with more efficient transformation key generation algorithm. Finally, we use the PBC library to test the efficiency of our schemes and compare the results with some previous ones, which shows that our schemes are more efficient in terms of decryption outsourcing and transformation key generation.
Zhao, Zhen, Lai, Jianchang, Susilo, Willy, Wang, Baocang, Hu, Yupu, Guo, Fuchun.  2019.  Efficient Construction for Full Black-Box Accountable Authority Identity-Based Encryption. IEEE Access. 7:25936—25947.
Accountable authority identity-based encryption (A-IBE), as an attractive way to guarantee the user privacy security, enables a malicious private key generator (PKG) to be traced if it generates and re-distributes a user private key. Particularly, an A-IBE scheme achieves full black-box security if it can further trace a decoder box and is secure against a malicious PKG who can access the user decryption results. In PKC'11, Sahai and Seyalioglu presented a generic construction for full black-box A-IBE from a primitive called dummy identity-based encryption, which is a hybrid between IBE and attribute-based encryption (ABE). However, as the complexity of ABE, their construction is inefficient and the size of private keys and ciphertexts in their instantiation is linear in the length of user identity. In this paper, we present a new efficient generic construction for full black-box A-IBE from a new primitive called token-based identity-based encryption (TB-IBE), without using ABE. We first formalize the definition and security model for TB-IBE. Subsequently, we show that a TB-IBE scheme satisfying some properties can be converted to a full black-box A-IBE scheme, which is as efficient as the underlying TB-IBE scheme in terms of computational complexity and parameter sizes. Finally, we give an instantiation with the computational complexity as O(1) and the constant size master key pair, private keys, and ciphertexts.
Khan, Aasher, Rehman, Suriya, Khan, Muhammad U.S, Ali, Mazhar.  2019.  Synonym-based Attack to Confuse Machine Learning Classifiers Using Black-box Setting. 2019 4th International Conference on Emerging Trends in Engineering, Sciences and Technology (ICEEST). :1—7.
Twitter being the most popular content sharing platform is giving rise to automated accounts called “bots”. Majority of the users on Twitter are bots. Various machine learning (ML) algorithms are designed to detect bots avoiding the vulnerability constraints of ML-based models. This paper contributes to exploit vulnerabilities of machine learning (ML) algorithms through black-box attack. An adversarial text sequence misclassifies the results of deep learning (DL) classifiers for bot detection. Literature shows that ML models are vulnerable to attacks. The aim of this paper is to compromise the accuracy of ML-based bot detection algorithms by replacing original words in tweets with their synonyms. Our results show 7.2% decrease in the accuracy for bot tweets, therefore classifying bot tweets as legitimate tweets.
Bartan, Burak, Pilanci, Mert.  2019.  Distributed Black-Box optimization via Error Correcting Codes. 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton). :246—252.
We introduce a novel distributed derivative-free optimization framework that is resilient to stragglers. The proposed method employs coded search directions at which the objective function is evaluated, and a decoding step to find the next iterate. Our framework can be seen as an extension of evolution strategies and structured exploration methods where structured search directions were utilized. As an application, we consider black-box adversarial attacks on deep convolutional neural networks. Our numerical experiments demonstrate a significant improvement in the computation times.
Song, Chengru, Xu, Changqiao, Yang, Shujie, Zhou, Zan, Gong, Changhui.  2019.  A Black-Box Approach to Generate Adversarial Examples Against Deep Neural Networks for High Dimensional Input. 2019 IEEE Fourth International Conference on Data Science in Cyberspace (DSC). :473—479.
Generating adversarial samples is gathering much attention as an intuitive approach to evaluate the robustness of learning models. Extensive recent works have demonstrated that numerous advanced image classifiers are defenseless to adversarial perturbations in the white-box setting. However, the white-box setting assumes attackers to have prior knowledge of model parameters, which are generally inaccessible in real world cases. In this paper, we concentrate on the hard-label black-box setting where attackers can only pose queries to probe the model parameters responsible for classifying different images. Therefore, the issue is converted into minimizing non-continuous function. A black-box approach is proposed to address both massive queries and the non-continuous step function problem by applying a combination of a linear fine-grained search, Fibonacci search, and a zeroth order optimization algorithm. However, the input dimension of a image is so high that the estimation of gradient is noisy. Hence, we adopt a zeroth-order optimization method in high dimensions. The approach converts calculation of gradient into a linear regression model and extracts dimensions that are more significant. Experimental results illustrate that our approach can relatively reduce the amount of queries and effectively accelerate convergence of the optimization method.
Wu, Yi, Liu, Jian, Chen, Yingying, Cheng, Jerry.  2019.  Semi-black-box Attacks Against Speech Recognition Systems Using Adversarial Samples. 2019 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN). :1—5.
As automatic speech recognition (ASR) systems have been integrated into a diverse set of devices around us in recent years, security vulnerabilities of them have become an increasing concern for the public. Existing studies have demonstrated that deep neural networks (DNNs), acting as the computation core of ASR systems, is vulnerable to deliberately designed adversarial attacks. Based on the gradient descent algorithm, existing studies have successfully generated adversarial samples which can disturb ASR systems and produce adversary-expected transcript texts designed by adversaries. Most of these research simulated white-box attacks which require knowledge of all the components in the targeted ASR systems. In this work, we propose the first semi-black-box attack against the ASR system - Kaldi. Requiring only partial information from Kaldi and none from DNN, we can embed malicious commands into a single audio chip based on the gradient-independent genetic algorithm. The crafted audio clip could be recognized as the embedded malicious commands by Kaldi and unnoticeable to humans in the meanwhile. Experiments show that our attack can achieve high attack success rate with unnoticeable perturbations to three types of audio clips (pop music, pure music, and human command) without the need of the underlying DNN model parameters and architecture.
Zhao, Pu, Liu, Sijia, Chen, Pin-Yu, Hoang, Nghia, Xu, Kaidi, Kailkhura, Bhavya, Lin, Xue.  2019.  On the Design of Black-Box Adversarial Examples by Leveraging Gradient-Free Optimization and Operator Splitting Method. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). :121—130.
Robust machine learning is currently one of the most prominent topics which could potentially help shaping a future of advanced AI platforms that not only perform well in average cases but also in worst cases or adverse situations. Despite the long-term vision, however, existing studies on black-box adversarial attacks are still restricted to very specific settings of threat models (e.g., single distortion metric and restrictive assumption on target model's feedback to queries) and/or suffer from prohibitively high query complexity. To push for further advances in this field, we introduce a general framework based on an operator splitting method, the alternating direction method of multipliers (ADMM) to devise efficient, robust black-box attacks that work with various distortion metrics and feedback settings without incurring high query complexity. Due to the black-box nature of the threat model, the proposed ADMM solution framework is integrated with zeroth-order (ZO) optimization and Bayesian optimization (BO), and thus is applicable to the gradient-free regime. This results in two new black-box adversarial attack generation methods, ZO-ADMM and BO-ADMM. Our empirical evaluations on image classification datasets show that our proposed approaches have much lower function query complexities compared to state-of-the-art attack methods, but achieve very competitive attack success rates.
Taori, Rohan, Kamsetty, Amog, Chu, Brenton, Vemuri, Nikita.  2019.  Targeted Adversarial Examples for Black Box Audio Systems. 2019 IEEE Security and Privacy Workshops (SPW). :15—20.
The application of deep recurrent networks to audio transcription has led to impressive gains in automatic speech recognition (ASR) systems. Many have demonstrated that small adversarial perturbations can fool deep neural networks into incorrectly predicting a specified target with high confidence. Current work on fooling ASR systems have focused on white-box attacks, in which the model architecture and parameters are known. In this paper, we adopt a black-box approach to adversarial generation, combining the approaches of both genetic algorithms and gradient estimation to solve the task. We achieve a 89.25% targeted attack similarity, with 35% targeted attack success rate, after 3000 generations while maintaining 94.6% audio file similarity.
Tsingenopoulos, Ilias, Preuveneers, Davy, Joosen, Wouter.  2019.  AutoAttacker: A reinforcement learning approach for black-box adversarial attacks. 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS PW). :229—237.
Recent research has shown that machine learning models are susceptible to adversarial examples, allowing attackers to trick a machine learning model into making a mistake and producing an incorrect output. Adversarial examples are commonly constructed or discovered by using gradient-based methods that require white-box access to the model. In most real-world AI system deployments, having complete access to the machine learning model is an unrealistic threat model. However, it is possible for an attacker to construct adversarial examples even in the black-box case - where we assume solely a query capability to the model - with a variety of approaches each with its advantages and shortcomings. We introduce AutoAttacker, a novel reinforcement learning framework where agents learn how to operate around the black-box model by querying it, to effectively extract the underlying decision behaviour, and to undermine it successfully. AutoAttacker is a first of kind framework that uses reinforcement learning and assumes nothing about the differentiability or structure of the underlying function and is thus robust towards common defenses like gradient obfuscation or adversarial training. Finally, without differentiable output, as in binary classification, most methods cease to operate and require either an approximation of the gradient, or another approach altogether. Our approach, however, maintains the capability to function when the output descriptiveness diminishes.