The gradual deployment of self-driving cars will inevitably lead to the emergence of a new important class of cyber-physical-human systems where autonomous vehicles interact with human-driven vehicles via on-board sensors or vehicle-to-vehicle communications. Reinforcement learning along with control theory can help meet the safety requirements for real-time decision making and Level 5 autonomy in self-driving vehicles. However, it is widely known that conventional reinforcement learning policies are vulnerable to adversarial or non-adversarial perturbations to their observations, similar to adversarial examples for classifiers and/or reward (packet) drops of the learning. Such issues are exacerbated by concerns of addressing resiliency as the use of open communication and control platforms for autonomy becomes essential, and as the industry continues to invest in such systems. Decision making mechanisms, designed to incorporate agility with the help of reinforcement learning, allow self-adaptation, self-healing, and self-optimization. This research will contribute and unify the body of knowledge of several diverse fields including reinforcement learning, security, automatic control, and transportation for resilient autonomy with humans-in-the-loop.

In this project, to counter action and observation manipulation as well as reward drops, the principal investigators will leverage proactive switching policies that aim (i) to provide robustness to adversarial inputs and reward drops in the closed-loop reinforcement learning mechanisms, (ii) to increase the cost of manipulation by deception, (iii) to limit the exposure of vulnerable actions and observations, and (iv) to provide stability, optimality, and robustness guarantees. Ultimately, the investigators will develop fundamental contributions to each of the above-mentioned fields and amalgamate these fields to provide a unique synthesis framework. The outcomes of this project will increase levels of confidence in autonomous technologies from ethical perspectives by providing an underpinning for curtailing accidents. The proposed framework can be extended to other key enablers of the global economy, including smart and connected cities, healthcare, and networked actions of smart systems while decreasing environmental pollution and minimizing the adverse environmental impacts on human health. The project will train the next generation of students from various levels, ages, and cultures through well-coordinated, level appropriate involvement in research and educational activities while providing a unique opportunity for the students to appreciate efficient, autonomous, and low-cost designs. This project will also contribute to future engineering curricula, pursue a substantial integration of research and education, and provide opportunities to engage students from the underrepresented group.