This Cyber-Physical Systems (CPS) project will make foundational methodological advances that enable safe and robust reinforcement learning (RL)-based control algorithmic solutions that are driven by problems in smart traffic signal control systems. Recent advances in computation, communication, storage, and sensing have led to a demand for data-driven learning-based decision-making and control in modern cyber-physical systems (CPSs), such as smart transportation systems. In such systems, decision-making agents need to operate safely and in a robust manner while working in complex environments with constraints that need to be respected. This project will develop foundational advances in robust RL solutions, and safe and constrained RL with provable guarantees by taking traffic signal control systems within smart transportation systems as our motivating CPS application and evaluation platform. This work will additionally focus on advancing curriculum development, recruitment of students from under-represented groups, involvement of undergraduate students in research, K-12 outreach, and also research community outreach via workshops, conference sessions, and seminars. The researchers will interface with companies and other stakeholders to communicate the results of the research as well as provide them with educational material on methodology. 

The technical approaches include: 1. Robust RL solutions incorporating model class knowledge, use of future predictions and robustness characterizations, and off-policy methods to address distributional shifts and data paucity arising from the use of a simulator/emulator or offline data; and 2. Efficient, safe, and constrained RL algorithms using model-free approaches and function-approximated methods, and also methods for partially-observed systems. To close the loop with the motivating CPS application, the RL algorithms will be evaluated in the context of traffic signal control via a comprehensive simulation-based evaluation using models of two instrumented sites.

Self-driving cars and home assistants provide just a small glimpse of the future cost-costrainted complex cyber-physical-human systems (CPHS) that will integrate engineering systems with the natural word and humans. This project will devise new mathematical tools and methods to systematically describe CPHS and optimize their operation. The application focus is on wireless body area networks, a natural CPHS representative with humans in the loop, heavily resource-constrained operation, and heterogeneous components that are intertwined with and altered by human behavior. The end will result will help understand important factors related to the operation of CPHS and how to optimize their operation. It will advance stochastic modeling, estimation and control theories to collectively address the challenges associated with this problem. It will also expose underrepresented K-12 students in Albany city to STEM fields from the lens of CPHS through project-based school visits and hands-on workshops on simple sensor systems, and prepare UAlbany students to become CPHS innovators by introducing CPHS concepts and activities in existing courses. Women and underrepresented groups will be encouraged to participate in this project by leveraging existing minority and underrepresented groups programs at the University at Albany.

CPHS have the potential to adaptively optimize their operation towards continuous real-time monitoring of an individual's state, environment and related behaviors, while providing real-time recommendations. To unleash the potential of CPHS, unique challenges related to sensing, communication, computation and control need to be jointly addressed in the presence of heterogeneous data, resource constraints and humans-in-the-loop. This fundamental research will advance cyber-physical systems science by devising new mathematical tools and methods, and a theoretical framework that can be used as a building block for various CPHS applications. Specifically, the project will (i) devise a new theoretical stochastic model to describe key CPHS variables and their interactions, (ii) design novel accurate and scalable estimators for CPHS and derive relevant theoretical performance bounds to quantify the fundamental limits of the estimation process in this context, and (iii) devise new controlled sensing, commmunication, and recommendations strategies to optimize the operation of systems with complex dynamics and heterogeneous capabilities.

This project will improve the ability to build artificial intelligence algorithms for Cyber-Physical Systems (CPS) that incorporate communications technologies by developing methods of learning from simulation environments. The specific application area is connected and automated vehicles (CAV) that drive strategically to reduce stop-and-go traffic. Employing communication between vehicles can improve the efficiency of vehicle control systems to manage traffic compared to vehicles without communication. The research of this project will explore the simulation of CAVs and how we can improve their algorithms to reduce traffic congestion, with core technology developments that are applicable to homes, health, and smart and connected communities. Increasingly at the heart of CPS are artificial intelligence algorithms, which can be programmed using a simulation of how the system should operate in the real world. A major challenge is building a simulation that accurately captures the complexity of the system in question, and how it can be controlled. The project includes partners from Toyota and Nissan that support testbeds enabling the research and accelerate transition of research to practice. The project aslo includes state and local Government stakeholders / partners which will facilitate experimentation in the real-world and demonstration of traffic congestion objectives as well as potentially emission reduction. Tools, technologies, and datasets generated in this project will be shared as active resources to support access beyond the life of the project. The project brings a focus on mentorship for undergraduate researchers, in order to broaden participation in computing. 

This project will develop new reinforcement learning approaches for Lagrangian control that accommodate communication and networking between actuators. A motivating domain that will be an application area of the project is CAVs. A major challenge is leveraging a small number of CAVs before those technologies realize full adoption rates. Vehicle and infrastructure communication technologies can be more useful for congestion management when feeding into a group of sparse, coordinated Lagrangian control agents. The project will use data from existing traffic sensors and testbeds to drive learning and control development. A fleet of instrumented and controllable passenger vehicles will be used for data collection and actuation. Validation experiments will be conducted using these vehicles on live roadways, and the results will be validated using a camera-based testbed that collects detailed traffic data.

This NSF project aims to develop a novel computational framework for informed contextual multi-armed bandits (iCMABs) that will be capable of robustly operating in complex, time-varying environments. The project will bring transformative change to the way that intelligent decision-making agents are designed for CPS, specifically those that utilize variants of multi-armed bandits. The intellectual merits of the project include: I) designing novel informed contextual bandits that maintain a generative model of their external world/environment, II) designing neural architecture search processed based on neuro-evolution to automatically design these generative models in an online manner, III) providing mechanisms to identify and address corrupt contextual and reward information, and IV) facilitating a process that enables the agent to generate predictions over longer-term horizons by querying its internal generative model. The broader impacts of the project include: I) advancing intelligent CPS through the iCMAB framework, II) providing decision-making modules and processes that readily integrate with many intelligent CPS/operations, and III) making important contributions to the field of machine learning and nature-inspired computing, specifically the automated design of intelligent agents based on artificial neural networks (ANNs). 

Current challenges faced by intelligent CPS, such as those used for tasks such as sensor validation and activity decisions, include being required to robustly operate in the face of noise, coming from things such as corrupted, fragmented, and uncertain reward values and context/state information, as well as having to adapt and make predictions in real-time and continually. Our proposed iCMAB framework will enable CPS to tackle these problems by: I) jointly evolving, in an online fashion, a reward forecasting model and a generative world model of contexts, II) providing a measure of confidence in predictions of both reward and context signals, and III) utilizing evolving recurrent neural networks (eRNNs) and brain-inspired neural systems/mechanisms to predict both contextual and reward information in the streaming data setting while mitigating catastrophic forgetting. This updated information will serve as input to CPS that are driven by contextual-bandits, enabling them to take more informed actions.

This project will provide funding for students from U.S. institutions of higher learning to attend the 2023 Cyber-Physical Systems and Internet-of-Things Week (CPS-IoT Week 2023) in San Antonio, Texas during May 9-12, 2023. This is the first in-person CPS IoTT week since the pandemic in 2020. CPS-IoT Week (previously known as CPSWeek) is the premier conference for Cyber-Physical Systems research, and is comprised of five flagship conferences in CPS, namely ICCPS (International Conference on Cyber-Physical Systems), RTAS (Real-Time and Embedded Technology and Applications Symposium), IPSN (Information Processing in Sensor Networks), HSCC (Hybrid Systems: Computation and Control) and IoTDI (Internet of Things Design and Implementation). In addition to these 5 conferences, there are multiple workshops and demonstrations that permit papers to be submitted and presented by graduate students and postdoctoral researchers. 

Cyber-physical systems (CPS) are engineered systems that are built from, and depend upon, the seamless integration of computation and physical components. Advances in CPS will enable capability, adaptability, scalability, resiliency, safety, security, and usability that will expand the horizons of these critical systems. CPS technologies are transforming the way people interact with engineered systems, just as the Internet has transformed the way people interact with information. Travel support to this conference for today's graduate students and postdoctoral researchers enables broad participation by tomorrow's researchers in this interdisciplinary field.

Fishes are masters of locomotion in fluids owing to their highly integrated biological sensing, computing and motor systems. They are adept at collecting and exploiting rich information from the surrounding fluids for underwater sensing and locomotion control. Inspired and informed by fish swimming, this research aims to develop a novel bio-inspired cyber-physical system (CPS) that integrates the ?physical? robot fish and fluid environment with the ?cyber? robot control & machine learning algorithms. Specifically, this CPS system includes i) a pressure sensory skin with distributed sensing capability to collect flow information, ii) control and learning algorithms that compute robot motor signals, output by central pattern generators (CPGs) which receive pressure sensory feedback, iii) a robot fish platform to implement and validate the CPS framework for underwater sensing and control tasks, and iv) experimental and computational methods to investigate and model the underlying fluid physics. This CPS system will have immediate impacts on the core CPS research areas such as design, control, data analytics, autonomy, and real-time systems. It will also significantly impact a wide range of engineering applications which demand distributed sensing, control and adaptive actuation. Examples include human-machine interactions, medical robots, unmanned aerial/underwater vehicles, drug dosing, medical therapeutics, and space deployable structures among others. Leveraging the multidisciplinary nature of this research, this award will support a variety of educational and outreach activities. In particular, a list of activities in broadening participation in engineering will be carried out.

This research project integrates multiple CPS technologies to develop bio-inspired technologies for swarm control of fish. These include inthanovations in a pressure sensitive skin project will first develop a distributed pressure sensitive synthetic skin, which will be installed on robotic fishes to map the pressure distribution on their body and caudal-fin surfaces. The distributed pressure information will then be used in a feedback control policy that modulates CPGs to produce caudal-fin motion patterns of the robotic fishes. The control policy and the caudal-fin motion patterns will be optimized via reinforcement learning first in a surrogate fluid environment and then in the true fluid environment. The surrogate fluid environment will be developed using data-driven non-parametric models informed by physics-based hydrodynamic models of fish swimming, trained using combined experimental and Computational Fluid Dynamics (CFD) simulation data. The above control-learning methods will also be used to achieve efficient schooling in a group of robotic fishes, individually controlled by a CPG, which interacts with each other through surrounding fluids and pressure sensory feedback. The optimized swimming/schooling performance of robotic fishes and the underlying physics will be studied using CFD simulation. Together, this research will advance CPS knowledge on: 1) the design and creation of electronic and sensor materials and devices for robot skin applications; 2) the development of data-efficient, physics-informed learning methods for robotic systems that operate in complex environments, especially leveraging the recent progress on deep learning to exploit the spatial and temporal richness of the pressure data for underwater sensing and robot control; and 3) the flow physics and modeling of fish swimming.

Propelled by the growth in demand for artificial intelligence-enabled applications, the past decade has witnessed the emergence of Collaborative Cyber-Physical Learning Systems (CCPLS). CCPLS carry out distributed, learning-based processing tasks through coordination among Cyber-Physical System (CPS) devices, and are envisioned to provide critical functionality across the commercial and defense sectors in the next several years. However, the data generated by CCPLS is often large-scale, high-dimensional, heterogeneous, and time-varying, which poses critical challenges for intelligence modeling. Concurrently, unmanned vehicles, in particular Unmanned Aerial Vehicles (UAVs), have shown promise of scaling up information-sharing in CCPLS, especially in under-served regions such as rural areas. The project's novelties are in establishing a concrete foundation for UAV-CCPLS integration that unifies the associated learning, networking, and communication design aspects around appropriate intelligence metrics. The project's impacts are the development of UAV-assisted CCPLS for smart agriculture tasks, as well as advancing the manufacturing of UAVs and other unmanned vehicles tailored for CCPLS. Project outcomes will be disseminated by releasing open-source software and research videos and tutorials. The investigators will further engage in Curriculum development, diversity, and outreach activities including mentoring undergraduate researchers. 

Research investigations center around three interconnected thrusts. Thrust 1 develops a novel UAV-assisted intelligence framework for CCPLS and introduces a precise, task-oriented notion of data dynamics and heterogeneity. Additionally, this thrust develops a `learning for learning? framework that aims to predictively estimate the rate of data dynamics. Thrust 2 investigates methodologies for jointly optimizing resource utilization and intelligence quality through co-design of UAV trajectories and UAV-to-CPS network establishment. The data dynamics framework from Thrust 1 is integrated into this design through an online, network-aware sequential decision-making framework. Finally, Thrust 3 develops CCPLS communication protocols based on learning-aware uplink and downlink wireless beamformers and over-the-air aggregation methods. These protocols are tailored to the specific needs of the UAV-assisted learning systems, e.g., the transmission of noisy information over UAV-to-UAV and UAV-to-access point communication links.

Many cyber-physical systems (CPS) have real-time (RT) requirements. For these RT-CPS, such as a network of unmanned aerial vehicles that deliver packages to customers? homes or a robot that performs/aides in cardiac surgery, deadline misses may result in economic losses or even fatal consequences. At the same time, as these RT-CPS interact with, and are depended on by, humans, they must also be trustworthy. The goal of this research is to design secure RT-CPS that are less complex, easier to analyze, and reliable for critical application domains such as defense, medicine, transportation, manufacturing, and agriculture, to name just a few. Since RT-CPS now permeate most aspects of our daily lives, especially in the smart city and internet-of-things (IoT) context, this research will improve confidence in automated systems by users. Research results will be disseminated to both academia and industry, and permit timely adoption since the hardware required in this research is already publicly available. This project will result in a pipeline of engineers and computer scientists who are well-versed in the interdisciplinary nature of securing RT-CPS, as well as course modules and red-teaming exercises for undergraduate students in all engineering disciplines and interactive learning modules and internship experience for K-12 students in D.C., Detroit, Dallas, and St. Louis.

The goal of this research is to design secure RT-CPS from the ground up while explicitly accounting for physical dynamics of said RT-CPS at runtime to achieve resilience via prevention and detection of, and recovery from, attacks. This will be accomplished by (i) securing the scheduling infrastructure from the ground up, (ii) using a formal framework for trading off security against timeliness while accounting for system dynamics, and for the cost of security to be explicitly quantified, and (iii) performing state- and function-dependent on-demand recovery. Said RT-CPS will be able to proactively prevent attacks using moving target defenses, as well as detect and recover from attacks that cannot be avoided. This research will pave the way for RT-CPS and internet-of-things (IoT) to be implemented with confidence: their timely and correct operation guaranteed. Specific contributions of this research are: (i) a trusted scheduling infrastructure that can protect the integrity of the real-time tasks, the scheduler, its task queues, and I/O, and which can recover from (intentional) errors, (ii) a probabilistic real-time/security co-design framework that exploits trusted execution to protect the security of the real-time tasks, (iii) novel schedulability analysis techniques, (iv) an incremental recovery mechanism for continuous operation, and (v) validation on automated ground vehicles, drones, and robot arms. Contributions expanding the knowledge base will be made to the fields of CPS, IoT, real-time systems, security, and control systems.

Systematic modeling, analysis and design of social, economic and technological systems requires addressing myriad tradeoffs that natural systems and human groups handle so effectively. A confluence of ideas from diverse disciplines, including arts, ecology, neuroscience, and control theory will further enhance our understanding of the underpinnings of such remarkable systems and enable us to facilitate such robustness and resilience in engineered systems. Examples of such systems range from electric networks to connected and autonomous vehicles to smart manufacturing to mixed human-automata systems. Broader issues that require such confluence include climate change, energy efficiency, and human-wildlife conflicts. To begin a dialogue aimed at improving synergistic approaches to these challenges, the proposed workshop on Convergence with Control: Bridging the Arts, Ecology, Neuroscience, and Engineering aims to bring a diverse scientific community together and foster fruitful cross-disciplinary collaborations. It will feature diverse and distinguished domestic and international speakers, including early career researchers. The participants will be from cross-cutting disciplines, including the arts, ecology, neuroscience, and engineering. In addition to the talks, the workshop will feature several activities with cross-disciplinary themes that will facilitate team-forming and facilitate deep scientific discussions. The workshop will consist of presentations by invited speakers interleaved with cross-disciplinary activities.

The purpose of the workshop is to facilitate and enhance research discussions and collaborations among experts in a broad range of disciplines, including biology, arts, and engineering. The invited speakers are leading experts in the areas of ecology, neuroscience, music and dance, dynamical systems, and control theory. The sub-themes, (i) collective behavior and control, (ii) robotics, learning and control, and (iii) neuroscience and control, will ensure that the broad agenda of the workshop is focused both on theory and application. The invited speakers have been selected to ensure a broad representation of research disciplines and such exposure will also benefit young researchers immensely.
 

Emerging mobility systems, e.g., connected and automated vehicles and shared mobility, provide the most intriguing opportunity for enabling users to better monitor transportation network conditions and make better decisions for improving safety and transportation efficiency. However, different levels of vehicle automation in the transportation network can significantly alter transportation efficiency metrics (travel times, energy, environmental impact). Moreover, we anticipate that efficient transportation might alter human travel behavior causing rebound effects, e.g., by improving efficiency, travel cost is decreased, hence willingness-to-travel is increased. The latter would increase overall vehicle miles traveled, which in turn might negate the benefits in terms of energy and travel time. The project will consolidate emerging mobility systems and modes with real-world data and processed information leading to an equitable transportation system with broad economic, environmental, and societal benefits. We expect the outcome of this project to enhance our understanding of the rebound effects, changes in travel demand and capacity, human reception, adoption, and use of emerging mobility systems. 

The outcome of this research will deliver an online learning framework that will aim at distributing travel demand in a given transportation network resulting in a socially-optimal mobility system that travelers would be willing to accept. A ?socially-optimal mobility system? is defined as a mobility system that (1) is efficient (in terms of energy consumption and travel time), (2) does not cause rebound effects, and (3) ensures equity in transportation. The framework will establish new approaches in optimally controlling cyber-physical systems by merging learning and control approaches. It includes the development of new methods to enhance accessibility, safety, and equity in transportation and travelers? acceptance. In the context of the proposed framework, a ?social planner? faces the problem of aggregating the preferences of the travelers into a collective, system-wide decision when the private information of the travelers is not publicly known. Mechanism design theory will be used to derive the optimal routes and the selection of a transportation mode for all travelers so as to maximize accessibility, safety, and equity in transportation and travelers? acceptance. Online learning algorithms for contextual bandit problems will be developed to identify traveler preferences and to determine how they would respond to the social planner?s recommendations on routing and selection of a transportation mode.