The gradual deployment of self-driving cars will inevitably lead to the emergence of a new important class of cyber-physical-human systems where autonomous vehicles interact with human-driven vehicles via on-board sensors or vehicle-to-vehicle communications. Reinforcement learning along with control theory can help meet the safety requirements for real-time decision making and Level 5 autonomy in self-driving vehicles. However, it is widely known that conventional reinforcement learning policies are vulnerable to adversarial or non-adversarial perturbations to their observations, similar to adversarial examples for classifiers and/or reward (packet) drops of the learning. Such issues are exacerbated by concerns of addressing resiliency as the use of open communication and control platforms for autonomy becomes essential, and as the industry continues to invest in such systems. Decision making mechanisms, designed to incorporate agility with the help of reinforcement learning, allow self-adaptation, self-healing, and self-optimization. This research will contribute and unify the body of knowledge of several diverse fields including reinforcement learning, security, automatic control, and transportation for resilient autonomy with humans-in-the-loop.
In this project, to counter action and observation manipulation as well as reward drops, the principal investigators will leverage proactive switching policies that aim (i) to provide robustness to adversarial inputs and reward drops in the closed-loop reinforcement learning mechanisms, (ii) to increase the cost of manipulation by deception, (iii) to limit the exposure of vulnerable actions and observations, and (iv) to provide stability, optimality, and robustness guarantees. Ultimately, the investigators will develop fundamental contributions to each of the above-mentioned fields and amalgamate these fields to provide a unique synthesis framework. The outcomes of this project will increase levels of confidence in autonomous technologies from ethical perspectives by providing an underpinning for curtailing accidents. The proposed framework can be extended to other key enablers of the global economy, including smart and connected cities, healthcare, and networked actions of smart systems while decreasing environmental pollution and minimizing the adverse environmental impacts on human health. The project will train the next generation of students from various levels, ages, and cultures through well-coordinated, level appropriate involvement in research and educational activities while providing a unique opportunity for the students to appreciate efficient, autonomous, and low-cost designs. This project will also contribute to future engineering curricula, pursue a substantial integration of research and education, and provide opportunities to engage students from the underrepresented group.
Kyriakos G. Vamvoudakis was born in Athens, Greece. He received the Diploma (a 5-year degree, equivalent to a Master of Science) in Electronic and Computer Engineering from the Technical University of Crete, Greece in 2006 with highest honors. After moving to the United States of America, he studied at The University of Texas at Arlington with Frank L. Lewis as his advisor, and he received his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011 respectively. From May 2011 to January 2012, he was working as an Adjunct Professor and Faculty Research Associate at the University of Texas at Arlington and at the Automation and Robotics Research Institute. During the period from 2012 to 2016 he was project research scientist at the Center for Control, Dynamical Systems and Computation at the University of California, Santa Barbara. He was an assistant professor at the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech until 2018.
He currently serves as the Dutton-Ducoffe Endowed Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He holds a secondary appointment in the School of Electrical and Computer Engineering. His expertise is in reinforcement learning, control theory, game theory, cyber-physical security, bounded rationality, and safe/assured autonomy.
Dr. Vamvoudakis is the recipient of a 2019 ARO YIP award, a 2018 NSF CAREER award, a 2018 DoD Minerva Research Initiative Award, a 2021 GT Chapter Sigma Xi Young Faculty Award and his work has been recognized with best paper nominations and several international awards including the 2016 International Neural Network Society Young Investigator (INNS) Award, the Best Paper Award for Autonomous/Unmanned Vehicles at the 27th Army Science Conference in 2010, the Best Presentation Award at the World Congress of Computational Intelligence in 2010, and the Best Researcher Award from the Automation and Robotics Research Institute in 2011. He is a member of Tau Beta Pi, Eta Kappa Nu, and Golden Key honor societies and is listed in Who's Who in the World, Who's Who in Science and Engineering, and Who's Who in America. He has also served on various international program committees and has organized special sessions, workshops, and tutorials for several international conferences. He currently is a member of the Technical Committee on Intelligent Control of the IEEE Control Systems Society, a member of the Technical Committee on Adaptive Dynamic Programming and Reinforcement Learning of the IEEE Computational Intelligence Society, a member of the IEEE Control Systems Society Conference Editorial Board, an Associate Editor of: Automatica; IEEE Transactions on Automatic Control; IEEE Transactions on Neural Networks and Learning Systems; IEEE Computational Intelligence Magazine; IEEE Transactions on Systems, Man, and Cybernetics: Systems; IEEE Transactions on Artificial Intelligence; Neurocomputing; Journal of Optimization Theory and Applications; and of Frontiers in Control Engineering-Adaptive, Robust and Fault Tolerant Control. He had also served as a Guest Editor for, IEEE Transactions on Automation Science and Engineering (Special issue on Learning from Imperfect Data for Industrial Automation); IEEE Transactions on Neural Networks and Learning Systems (Special issue on Reinforcement Learning Based Control: Data-Efficient and Resilient Methods); IEEE Transactions on Industrial Informatics (Special issue on Industrial Artificial Intelligence for Smart Manufacturing); and IEEE Transactions on Intelligent Transportation Systems (Special issue on Unmanned Aircraft System Traffic Management). He is also a registered Electrical/Computer engineer (PE), a member of the Technical Chamber of Greece, an Associate Fellow of AIAA, and a Senior Member of IEEE.