This project expands how reinforcement learning frameworks can be used for Cyber-Physical Systems (CPS) for autonomy. The research utilizes intermittent reinforcement, where a reward is not given every time the desired response is performed. This differs from traditional reinforcement learning mechanisms, in which a reward is given for each point during online training. What is novel in this framework is that it can demonstrate how reinforcement learning can be used when rare events, or noisy and adversarial data, can affect the training and performance of these algorithms. The work will be validated on collaborative road freight transport and collaborative robotics testbeds, through international partnerships with Sweden and the United Kingdom. The project includes activities that integrate high-school students into challenging problems in machine learning areas, motivated through drone racing competitions. The goal of this research is to expand foundational knowledge through deepened ties between the learning, control, game theory, and CPS communities. The approach is to, (i) unify new perspectives of learning in engineering with respect to resiliency, bandwidth efficiency, robustness, and other aspects that cannot be achieved with the state-of-the-art approaches; (ii) develop intermittent deep learning methods for CPS that can mitigate sensor attacks and can handle cases of limited sensing capabilities; (iii) incorporate nonequilibrium game-theoretic learning in CPS with components whose decision-making, rationality, and information usage are fundamentally different; and (iv) investigate ways to transfer learning to new platforms. The project's education and outreach component includes internships that will lead to technology transfer, summer camps with a special focus on reaching out to underrepresented minorities and women, and collaboration with institutions in Sweden and the United Kingdom through student exchange programs.
Kyriakos G. Vamvoudakis was born in Athens, Greece. He received the Diploma (a 5-year degree, equivalent to a Master of Science) in Electronic and Computer Engineering from the Technical University of Crete, Greece in 2006 with highest honors. After moving to the United States of America, he studied at The University of Texas at Arlington with Frank L. Lewis as his advisor, and he received his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011 respectively. From May 2011 to January 2012, he was working as an Adjunct Professor and Faculty Research Associate at the University of Texas at Arlington and at the Automation and Robotics Research Institute. During the period from 2012 to 2016 he was project research scientist at the Center for Control, Dynamical Systems and Computation at the University of California, Santa Barbara. He was an assistant professor at the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech until 2018.
He currently serves as the Dutton-Ducoffe Endowed Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He holds a secondary appointment in the School of Electrical and Computer Engineering. His expertise is in reinforcement learning, control theory, game theory, cyber-physical security, bounded rationality, and safe/assured autonomy.
Dr. Vamvoudakis is the recipient of a 2019 ARO YIP award, a 2018 NSF CAREER award, a 2018 DoD Minerva Research Initiative Award, a 2021 GT Chapter Sigma Xi Young Faculty Award and his work has been recognized with best paper nominations and several international awards including the 2016 International Neural Network Society Young Investigator (INNS) Award, the Best Paper Award for Autonomous/Unmanned Vehicles at the 27th Army Science Conference in 2010, the Best Presentation Award at the World Congress of Computational Intelligence in 2010, and the Best Researcher Award from the Automation and Robotics Research Institute in 2011. He is a member of Tau Beta Pi, Eta Kappa Nu, and Golden Key honor societies and is listed in Who's Who in the World, Who's Who in Science and Engineering, and Who's Who in America. He has also served on various international program committees and has organized special sessions, workshops, and tutorials for several international conferences. He currently is a member of the Technical Committee on Intelligent Control of the IEEE Control Systems Society, a member of the Technical Committee on Adaptive Dynamic Programming and Reinforcement Learning of the IEEE Computational Intelligence Society, a member of the IEEE Control Systems Society Conference Editorial Board, an Associate Editor of: Automatica; IEEE Transactions on Automatic Control; IEEE Transactions on Neural Networks and Learning Systems; IEEE Computational Intelligence Magazine; IEEE Transactions on Systems, Man, and Cybernetics: Systems; IEEE Transactions on Artificial Intelligence; Neurocomputing; Journal of Optimization Theory and Applications; and of Frontiers in Control Engineering-Adaptive, Robust and Fault Tolerant Control. He had also served as a Guest Editor for, IEEE Transactions on Automation Science and Engineering (Special issue on Learning from Imperfect Data for Industrial Automation); IEEE Transactions on Neural Networks and Learning Systems (Special issue on Reinforcement Learning Based Control: Data-Efficient and Resilient Methods); IEEE Transactions on Industrial Informatics (Special issue on Industrial Artificial Intelligence for Smart Manufacturing); and IEEE Transactions on Intelligent Transportation Systems (Special issue on Unmanned Aircraft System Traffic Management). He is also a registered Electrical/Computer engineer (PE), a member of the Technical Chamber of Greece, an Associate Fellow of AIAA, and a Senior Member of IEEE.