This project expands how reinforcement learning frameworks can be used for Cyber-Physical Systems (CPS) for autonomy. The research utilizes intermittent reinforcement, where a reward is not given every time the desired response is performed. This differs from traditional reinforcement learning mechanisms, in which a reward is given for each point during online training. What is novel in this framework is that it can demonstrate how reinforcement learning can be used when rare events, or noisy and adversarial data, can affect the training and performance of these algorithms.
Kyriakos G. Vamvoudakis was born in Athens, Greece. He earned his Diploma in Electronic and Computer Engineering (equivalent to a Master of Science) from the Technical University of Crete, Greece, in 2006, graduating with highest honors. After relocating to the United States, he pursued further studies at The University of Texas at Arlington under the guidance of Frank L. Lewis, obtaining his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011, respectively. From May 2011 to January 2012, he served as an Adjunct Professor and Faculty Research Associate at the University of Texas at Arlington and the Automation and Robotics Research Institute. Between 2012 and 2016, he was a project research scientist at the Center for Control, Dynamical Systems, and Computation at the University of California, Santa Barbara. He then joined the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech as an assistant professor, a position he held until 2018.
He currently serves as the Dutton-Ducoffe Endowed Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He holds a secondary appointment in the School of Electrical and Computer Engineering. His expertise is in reinforcement learning, control theory, game theory, cyber-physical security, bounded rationality, and safe/assured autonomy.
He has received numerous prestigious awards, including the 2019 ARO YIP Award, the 2018 NSF CAREER Award, the 2018 DoD Minerva Research Initiative Award, and the 2021 GT Chapter Sigma Xi Young Faculty Award. His work has also been recognized with several best paper nominations and international awards, such as the 2016 International Neural Network Society Young Investigator (INNS) Award, the Best Paper Award for Autonomous/Unmanned Vehicles at the 27th Army Science Conference in 2010, the Best Presentation Award at the World Congress of Computational Intelligence in 2010, and the Best Researcher Award from the Automation and Robotics Research Institute in 2011. Dr. Vamvoudakis has served on various international program committees and has organized special sessions, workshops, and tutorials for several international conferences. He is the Editor-in-Chief of Aerospace Science and Technology and currently serves on the IEEE Control Systems Society Conference Editorial Board. Additionally, he is an Associate Editor for several journals, including Automatica, IEEE Transactions on Automatic Control, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Artificial Intelligence, Neural Networks, and the Journal of Optimization Theory and Applications. He is also a Senior Guest Editor for the IEEE Open Journal of Control Systems for the special issue on the intersection of machine learning with control. Previously, Dr. Vamvoudakis has served as a Guest Editor for various special issues, including those in IEEE Transactions on Automation Science and Engineering, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Industrial Informatics, and IEEE Transactions on Intelligent Transportation Systems. He is a registered Professional Engineer (PE) in Electrical/Computer Engineering, a member of the Technical Chamber of Greece, an Associate Fellow of AIAA, and a Senior Member of IEEE.
Participatory science has opened opportunities for many to participate in data collection for science experiments about the environment, local transportation, disaster response, and public safety where people live. The nature of the collection by non-scientists on a large scale carries inherent risks of sufficient coverage, accuracy and reliability of measurements. This project is motivated by the challenges in data and predictive analytics and in control for participatory science data collection and curation in cyber-physical systems (CPS) experiments.
Good algorithmic foundations for flight planning on the scale required for managing dense urban drone traffic we can expect to see in the future are currently still missing. This project provides prototype algorithms for managing this dense drone traffic. The project develops a concept for a coordination system that is able to find collision-free paths for a large number of flying unmanned air vehicles of different size and capability. It uses a hierarchical approach, combining centralized and local coordination, to manage complexity for a large-scale problem.
Complex engineered systems that can adapt to their environments while maintaining safety guarantees are crucial in many applications including Internet-of-Things, transportation, and electric power systems. The primary objective of this project is to develop a scalable design methodology to control very large collections of systems to achieve common objectives despite cyber and physical constraints.