This project investigates new reinforcement learning (RL) approaches for cyber-physical autonomy to bridge the gap between current intelligent systems and human-level intelligence. The nature of many cyber-physical systems (CPS) is distributed, heterogeneous, and high-dimensional, making the hand-coded functions and task-specific information hard to design in the learning scheme. Large amount of training data is often required for achieving the desired performance, however this limits the generalization to other tasks. Hence, this project is to explore the new RL strategies to enable CPS with the capabilities of autonomous learning and generalization to rapidly adapt in unknown situations that were not assumed in the design phase. The results are expected to transform how agents interact in high-dimensional and heterogeneous environment, and therefore could potentially provide in-depth findings for exploring creativity in frontier Artificial Intelligence techniques.

Emerging technologies in communications and vehicle technologies will allow future autonomous vehicles to be platooned together with wireless communications (cyber-connected) or physically forming an actual train (physically-connected). When physically connected, vehicles may dock to and undock from each other en-route when vehicles are still moving. While such platooning can potentially offer substantial societal benefits in safety, mobility and environmental friendliness, their emergence also challenges the classic traffic flow models that do not account for the state that vehicles can have very short to no gaps from each other. And yet, classic traffic flow models are being used for all traffic simulations for assessment on safety, mobility and environment. This project aims to expand classic highway traffic flow models to account for states where vehicles can be very close to or even physically connected with each other. These new models will help stakeholders plan and manage future transportation systems and supply the engineering curriculum with new methods, tools, and experimental platforms oriented towards future smart urban systems.

This Cyber-Physical Systems (CPS) grant will focus on the development of an urban traffic management system, which is driven by public needs for improved safety, mobility, and reliability within metropolitan areas. Future cities will be radically transformed by the Internet of Things (IoT), which will provide ubiquitous connectivity between physical infrastructure, mobile assets, humans, and control systems. In particular, IoT and smart traffic management have the potential to significantly improve increasingly faltering transportation systems that account for over 25% of greenhouse gas emissions and over one trillion dollars of annual economic and social loss. The project develops a hybrid twin that operates in parallel with the real world at real-time resolution, leveraging machine learning and edge computing, to monitor surrounding traffic, send safety warnings to connected vulnerable users, and provide learning-based controls to traffic lights and automated vehicles. As such, the broader impacts include advancing the understanding of urban traffic modeling, computation, and simulation, and enriching transportation science with data science. The accompanying educational plan aims to broaden participation in computing and engineering by underrepresented minorities and women via outreach programs, including programs for Harlem public school teachers and K-12 students, as well as new graduate course development.

Cyber-physical systems such as self-driving cars, drones, and intelligent transportation rely heavily on machine learning techniques for ever-increasing levels of autonomy. In the example of autonomous vehicles, deep learning or deep neural networks can be employed for perception, sensor fusion, prediction, planning, and control tasks. However powerful such machine learning techniques have become, they also expose a new attack surface, which may lead to vulnerability to adversarial attacks and potentially harmful consequences in security- and safety-critical scenarios. This project investigates adversarial machine learning challenges faced by autonomous cyber-physical systems with the aim of formulating defense strategies. The project will collaborate with the Center for STEM (Science, Technology, Engineering and Math) Education at Northeastern University and the Office of Access and Inclusion Center at University of California at Irvine to engage undergraduates, women, and minority students in independent research projects.

The goal of this next phase of the Cyber-Physical Systems Virtual Organization (CPS-VO) is to continue to support and renew the CPS-VO infrastructure, while embracing the data revolution. The project will expand the CPS-VO's capacity to disseminate and collect data, models, and results from existing CPS research projects. This will increase the impact of those projects by making their research more readily accessible to others, as well as providing a means by which issues of repeatability and replicability can be demonstrated. Outreach activities within the research community include workshops to gather research project results, and advanced tutorials that enable CPS researchers to address issues of repeatability and reproducibility. The project will empower the hundreds of authors of tools, experimental validation data sets and education material from the last decade of CPS research to appropriately capture, archive, and make their results broadly accessible from within the Virtual Organization. The project will provide research supervision and mentoring opportunities to undergraduates from underrepresented groups. These undergraduates will be trained in techniques for reproducibility of CPS data results, preparing them for a career in industry or academia. Intellectual Challenges addressed by this new phase of the CPS-VO include: (1) accelerating the pace at which the results of experimental validation of CPS research can be leveraged by other researchers, whether through physical system experiments or design studios; (2) using and replicating active resources and CPS design studios in the virtual era that is embraced by education and research groups; (3) expanding active resources to include data and data analysis resources, empowering CPS researchers to embrace the data revolution; (4) creating highly configurable service bundles that minimize the time investment for publishing tools, design studios, data and models on the CPS-VO platform; and (5) providing use case demonstrations at multiple scales of the CPS-VO modules, accelerating the pace at which adoption of the resources can occur.

Cancers are among the leading causes of death around the world, with an estimated annual mortality of close to 10 million. Despite significant efforts to develop effective cancer diagnosis and therapeutics, the ability to predict patient responses to anti-cancer therapeutic agents remains elusive. This is a critical milestone as getting the right choice of therapy early can mean superior anti-tumor outcomes and increased survival, while the wrong choice means tumor relapse, development of resistance, side effects without the desired benefit, and increased cost of treatment. An cyber-physical system that allows an accurate prediction of patient tumor responses to anti-cancer therapies; that is, enable real-time precision medicine, can have a transformative effect not only on health outcomes, but also on the costs of treatment. The goal of this project is therefore to develop an engineered cyber-physical system that combines advanced biological models with state-of-the-art artificial intelligence methods for predictive, automated screening of anti-cancer drugs and optimizations of their dosing. This will move science towards realizing the long-desired precision medicine paradigm leading to significant social impacts. The project has additional social impacts, including minimizing the exponentially growing ethical issues surrounding the use of animals in the past years through increased adoption of the engineered human cancer and heart tissue model systems. The project will provide opportunities to promote STEM education for K-12 students, train students, especially those from under-represented groups, and disseminate science and engineering knowledge to the public.

Deep Neural Networks (DNN) enabled Cyber-Physical Systems (CPS) hold great promise for revolutionizing many industries, such as drones and self-driving cars. However, the current generation of DNN cannot provide analyzable behaviors and verifiable properties that are necessary for safety assurance. This critical flaw in purely data-driven DNN sometimes leads to catastrophic consequences, such as vehicle crashes linked to self-driving and driver-assistance technologies. On the other hand, physics-model-based engineering methods provide analyzable behaviors and verifiable properties, but do not match the performance of DNN systems. These considerations motivate the work in this project which aims at physics-model-based neural networks (NN) redesign, yielding HyPhy-DNN: a hybrid self-correcting physics-enhanced DNN framework. HyPhy-DNN will provide the performance of DNNs and the analyzability and verifiability of physical models, thus providing a foundation for verifiably safe self-driving cars, drones, and other CPS systems. Moreover, the HyPhy-DNN will fundamentally advance the integration of deep learning and robust control to enable safety- and time-critical CPS to safely operate with high performance in unforeseen and dynamic environments.

This NSF CPS project aims to develop new techniques for modeling cyber-physical systems that will address fundamental challenges associated with scale and complexity in modern engineering. The project will transform human interaction with complex cyber-physical and engineered systems, including critical infrastructure such as interconnected energy networks. This will be achieved through a novel combination of data-driven techniques and physics-based approaches to give mathematical and computational models that are at once abstract enough to be understood by humans making key engineering decisions and precise enough to make quantitative predictions. The intellectual merits of the project include a novel confluence of emerging data science and model-analysis methods, including manifold learning and information geometry. The broader impacts of the project include the training of undergraduates, including those from underrepresented communities, several outreach activities, and publicly available open-source software.

Enhancing resource availability, health, security, and a sense of well-being can be enhanced by our ability to sense, analyze, and act on our world with efficient, safe, and secure engineered systems. To realize such systems requires a deep understanding of the interfaces between the cyber and physical worlds, leading to the establishment of the field of Cyber Physical Systems (CPS). While CPS as a discipline and application-enabler has evolved tremendously over the past decade, current graduate training does not sufficiently prepare students for fundamental discovery and innovation in CPS nor for effective translation of research to application development. This National Science Foundation Research Traineeship (NRT) award to the University of Virginia (UVA) will address this need by training graduate students to pursue fundamental CPS discovery and innovation and to collaborate with application domain experts to realize a smarter planet, specifically in the areas of smart health, smart cities, and autonomous systems. The traineeship anticipates training one hundred fifty-eight (158) master's and doctoral students, including thirty-three (33) funded PhD trainees, from civil and environmental engineering, computer science, electrical and computer engineering, mechanical and aerospace engineering, and systems and information engineering. Current CPS graduate training is lacking in three critical ways that will be addressed in this project. First, current courses do not instill the integrative knowledge needed for new scientific discovery and translational applications in the field of CPS. Second, most students do not have a sufficiently robust experience of convergence activities as part of their training. This lack exists not only in traditional engineering and computing education but also extends into analyses of target application domains and associated grand challenges. Third, explicit professional development is absent in most graduate training. Such exposure is critical in CPS given the field's potent role in our ever-evolving smart world, and CPS practitioners must attend to social issues regarding ethics, safety, privacy, communication, and policy. This traineeship will address these issues and drive CPS graduate education nationwide by developing a novel, comprehensive graduate training program that involves orientation, normalization modules, elective courses, experiential convergence research activities, professional development workshops, and a hands-on, testbed-driven educational curriculum. Students will acquire both the technical depth and the integrative transdisciplinary understanding of CPS and its associated application domains to be successful in CPS-related careers. During their time in the program, trainees will engage in transdisciplinary CPS research on projects related to challenges in smart health, smart cities, and autonomous systems. These research efforts have the potential for significant scientific and application impact, such that a smarter world can be achieved and associated societal grand challenges can be addressed. The NSF Research Traineeship (NRT) Program is designed to encourage the development and implementation of bold, new potentially transformative models for STEM graduate education training. The program is dedicated to effective training of STEM graduate students in high priority interdisciplinary research areas through comprehensive traineeship models that are innovative, evidence-based, and aligned with changing workforce and research needs.

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). This project will develop novel modeling, control, and optimization methods for connected and automated vehicles to operate in human-dominated traffic to improve the efficiency and sustainability of the urban transportation system while respecting individual drivers? unique behaviors and social norms accordingly. The significance of the research is highlighted by the following two needs. First, the inefficiency of the urban transportation system has resulted in substantial fuel waste and emissions over the decades. Leveraging vehicles? growing autonomy and connectivity, a significant boost of energy efficiency, emission performance, and traffic management can be achieved through dedicated control and optimization of vehicle maneuvers and routes. Second, human drivers will remain the majority of operators on the road in the foreseeable future. The resulting mixed traffic where connected and automated vehicles and human drivers share the road with frequent interactions requires detailed modeling of human drivers? behaviors in a socially compatible context. The proposed research can generate socioeconomic incentives such as improving the efficiency of the urban transportation system and promoting technology acceptance for sustainable mobility, thereby alleviating the nation's energetic and environmental concerns. The scientific outcome of the project will advance convergent research areas of control theory, optimization, human behavioral study, and machine learning. The project will involve an interdisciplinary team of students through hands-on research opportunities at Texas Tech University, which has been historically and actively engaged in serving the traditionally underrepresented student body in STEM, contributing towards equitable and inclusive educational and social outcomes.