This project expands how reinforcement learning frameworks can be used for Cyber-Physical Systems (CPS) for autonomy. The research utilizes intermittent reinforcement, where a reward is not given every time the desired response is performed. This differs from traditional reinforcement learning mechanisms, in which a reward is given for each point during online training. What is novel in this framework is that it can demonstrate how reinforcement learning can be used when rare events, or noisy and adversarial data, can affect the training and performance of these algorithms. The work will be validated on collaborative road freight transport and collaborative robotics testbeds, through international partnerships with Sweden and the United Kingdom. The project includes activities that integrate high-school students into challenging problems in machine learning areas, motivated through drone racing competitions. The goal of this research is to expand foundational knowledge through deepened ties between the learning, control, game theory, and CPS communities. The approach is to, (i) unify new perspectives of learning in engineering with respect to resiliency, bandwidth efficiency, robustness, and other aspects that cannot be achieved with the state-of-the-art approaches; (ii) develop intermittent deep learning methods for CPS that can mitigate sensor attacks and can handle cases of limited sensing capabilities; (iii) incorporate nonequilibrium game-theoretic learning in CPS with components whose decision-making, rationality, and information usage are fundamentally different; and (iv) investigate ways to transfer learning to new platforms. The project's education and outreach component includes internships that will lead to technology transfer, summer camps with a special focus on reaching out to underrepresented minorities and women, and collaboration with institutions in Sweden and the United Kingdom through student exchange programs.
Kyriakos G. Vamvoudakis was born in Athens, Greece. He earned his Diploma in Electronic and Computer Engineering (equivalent to a Master of Science) from the Technical University of Crete, Greece, in 2006, graduating with highest honors. After relocating to the United States, he pursued further studies at The University of Texas at Arlington under the guidance of Frank L. Lewis, obtaining his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011, respectively. From May 2011 to January 2012, he served as an Adjunct Professor and Faculty Research Associate at the University of Texas at Arlington and the Automation and Robotics Research Institute. Between 2012 and 2016, he was a project research scientist at the Center for Control, Dynamical Systems, and Computation at the University of California, Santa Barbara. He then joined the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech as an assistant professor, a position he held until 2018.
He currently serves as the Dutton-Ducoffe Endowed Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He holds a secondary appointment in the School of Electrical and Computer Engineering. His expertise is in reinforcement learning, control theory, game theory, cyber-physical security, bounded rationality, and safe/assured autonomy.
He has received numerous prestigious awards, including the 2019 ARO YIP Award, the 2018 NSF CAREER Award, the 2018 DoD Minerva Research Initiative Award, and the 2021 GT Chapter Sigma Xi Young Faculty Award. His work has also been recognized with several best paper nominations and international awards, such as the 2016 International Neural Network Society Young Investigator (INNS) Award, the Best Paper Award for Autonomous/Unmanned Vehicles at the 27th Army Science Conference in 2010, the Best Presentation Award at the World Congress of Computational Intelligence in 2010, and the Best Researcher Award from the Automation and Robotics Research Institute in 2011. Dr. Vamvoudakis has served on various international program committees and has organized special sessions, workshops, and tutorials for several international conferences. He is the Editor-in-Chief of Aerospace Science and Technology and currently serves on the IEEE Control Systems Society Conference Editorial Board. Additionally, he is an Associate Editor for several journals, including Automatica, IEEE Transactions on Automatic Control, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Artificial Intelligence, Neural Networks, and the Journal of Optimization Theory and Applications. He is also a Senior Guest Editor for the IEEE Open Journal of Control Systems for the special issue on the intersection of machine learning with control. Previously, Dr. Vamvoudakis has served as a Guest Editor for various special issues, including those in IEEE Transactions on Automation Science and Engineering, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Industrial Informatics, and IEEE Transactions on Intelligent Transportation Systems. He is a registered Professional Engineer (PE) in Electrical/Computer Engineering, a member of the Technical Chamber of Greece, an Associate Fellow of AIAA, and a Senior Member of IEEE.