Multi-agent coordination and collaboration is a core challenge of future cyber-physical systems as they start having more complex interactions with each other or with humans in homes or cities. One of the key challenges is that agents must be able to reason about and learn the behavior of other agents in order to be able to make decisions. This is particularly challenging because state of the art approaches such as recursive belief modeling over partner policies often do not scale. However, humans are very effective in coordinating and collaborating with each other without the need of any expensive recursive belief modeling. One hypothesis is that humans can effectively capture the sufficient representations required for coordinating on tasks. Similar to humans, the agents in a multi-agent setting can look for the sufficient statistics needed for coordination and collaboration. This project is about learning and approximating such sufficient statistics to enable effective collaboration and coordination. In addition, the investigators will study teaching and learning in settings where the agents have partial observation over the world and need to teach and learn from each other in order to achieve a collaborative task.

Important successful demonstrations of reinforcement learning for single agents have spurred the drive to determine whether such methods can extend to multiple agents. There have also been notable developments in the area of multi-agent systems, both in understanding the structure of the resulting interacting dynamics and in the development of practical reinforcement learning algorithms. The core objective of this project is: 1) the development of learning methods that approximate the well-known concept of sufficient statistics in multi-agent interactions; 2) the development of a reinforcement learning algorithm that leverages the representations of sufficient statistics for more effective planning, coordination, and collaboration in multi-agent settings; and 3) the development of algorithms that use the representations of sufficient statistics to enable teaching and learning in multi-agent settings under partial observation over the environment. The overall outcome of this project will be a new formalism along with algorithms, tools, and techniques that enhance multi-agent learning and control. The investigators will ground this in two main applications: 1) collaborative search and exploration and 2) collaborative transport of objects.

Participatory science has opened opportunities for many to participate in data collection for science experiments about the environment, local transportation, disaster response, and public safety where people live. The nature of the collection by non-scientists on a large scale carries inherent risks of sufficient coverage, accuracy and reliability of measurements. This project is motivated by the challenges in data and predictive analytics and in control for participatory science data collection and curation in cyber-physical systems (CPS) experiments.

This project focuses on data-driven frameworks to address these challenges in CPS-enabled participatory science that builds on statistics, optimization, control, natural language processing, CPS fundamentals, and coordination of participants, known as crowd steering. This framework, known as DCCDI for Data-driven Crowdsensing CPS Design and Implementation, tightly combines the underlying methods and techniques, especially focusing on physical sensors, mobility, and model-based approaches, to improve efficiency, effectiveness, and accountability. Validation of the DCCDI framework is conducted through simulations, case studies, and on real-world CPS-enabled experiments. This project closely integrates education and training with foundational research and public outreach that enhances interdisciplinary thinking about CPS systems, engages the public through participatory science, and broadens participation in science, technology, engineering, mathematics, and computer science.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Animal agriculture has intensified over the past several decades, and animals are managed increasingly as large groups. As animals are often located remotely on large expanses of pasture, continuous monitoring of animal health and well-being is labor-intensive and challenging. This project aims to develop a solar sensor-based smart farm Internet-of-Things network, which is versatile, reliable, and robust to cyberattacks for smart animal monitoring and to demonstrate its operation and practicality on real farms. The solar sensor network will leverage low-power, wide- area networking to enable animal care personnel to monitor the behavior and health of cattle remotely through the Internet. The proposed research will provide fundamental advances to building an energy-efficient, scalable, communication-efficient animal farm system, while ensuring high monitoring quality under uncertain, dynamic, and hostile smart farm environments. The success of this project will contribute to a farm management system by accurately observing, measuring and responding to variabilities in animal agriculture systems.

The proposed work will design an energy-centric solution that actively schedules communication and computation to minimize energy waste in energy harvesting contexts. The proposed sensor node will monitor biometrics, acceleration, and location of animals and is powered by solar energy. The proposal further builds a physical and medium-access communication layer that is actively aware of the energy-mismatch between the low-energy sensors and the more capable LoRa (Long Range) gateways. The adoption of wireless technologies introduces cybersecurity vulnerabilities, and hence, cybersecurity is another major design objective of the proposed system by leveraging belief models and deep learning techniques while maintaining high quality monitoring services. The proposed sensor network will be tested at Virginia Tech?s farm testbeds, which have been designed to test and showcase such technologies for pastured livestock. The research will also be beneficial to the fields of semiconductor devices, embedded systems, Internet-of-Things (IoT) devices, wireless communications including 5G and beyond, robust machine/deep learning, cybersecurity, statistical signal detection, and agricultural production. The project will pioneer transformative research to increase productivity of animal agriculture and allow for real-world testing of advancements.

Human blood clots kill an estimated 100,000 to 300,000 Americans each year. Current treatments rely on medications that break down clots, which can be combined with a surgical procedure that mechanically alters the clot. However, clot-busting medications and surgery are both linked to unintended adverse events. This project designs and studies miniature magnetic swimmers as a minimally invasive alternative to these treatments. These devices are millimeter-scale objects that have a helical shape and contain a small permanent magnet. A rotating magnetic field is used to remotely control the swimmers. The field makes the swimmer rotate and the helical shape converts the rotational movement into a propulsive force (much like a boat's propeller). The swimmers can be steered by changing the orientation of the applied magnetic field. They can precisely navigate within liquids along pre-defined 3D paths and could potentially navigate within the bloodstream toward a blood clot. The rotational movement can be used to abrade the clot and restore an appropriate blood flow.

This project introduces new biomedical paradigms and investigates the robust high-speed tetherless manipulation of magnetic swimmers in complex, time-varying environments. It requires multiple swimmer designs, innovative controllers, and integrated, dynamic sensing modalities. The project is divided into three synergetic objectives. First, a fluid-dynamics model for the swimmer must be created and validated experimentally. This model will be used to optimize the swimmer's geometry. Secondly, a robust swimmer controller will be built. The controller must be able to compensate for the variable blood flow present in the arteries. An ultrasound scanner will be used to track the position of the swimmer and obtain useful information about its environment. For example, the velocity of the swimmer can be used to infer the blood flow velocity. Finally, the feasibility of several surgical procedures will be studied via ex-vivo experiments. 3D navigation together with the removal of human blood clots will be performed inside a human heart phantom. Several swimmers end effectors will also be tested.

Innovations driven by recent progress in artificial intelligence (AI) have demonstrated human-competitive performance. However, as research expands to safety-critical applications, such as autonomous vehicles and healthcare treatment, the question of their safety becomes a bottleneck for the transition from theories to practice. Safety-critical autonomy must go through a rigorous evaluation before massive deployment. They are unique in the sense that failures may cause serious consequences, thus requiring an extremely low failure rate. This means that test results under naturalistic conditions are extremely imbalanced - with the failure cases being rare. The rarity, together with the complex AI structures, poses a huge challenge to design effective evaluation methods that cannot be adequately addressed by conventional methods.

This proposal aims to understand the fundamental challenges in assessing the risk of safety-critical AI autonomy and puts forward new theories and practical tools to develop certifiable, implementable, and efficient evaluation procedures. The specific aims of this research are to develop evaluation methods for three types of AI autonomy that cover a broad array of real-world applications: deep learning systems, reinforcement learning systems, and sophisticated systems comprising sub-modules, and validate them with the sensing and decision-making systems of real-world autonomous systems. This research lays the foundation for the PI?s long-term career goal to safely deploy AI in the physical world, opens up a new cross-cutting area to develop rigorous and efficient evaluation methods, addresses the urgent societal concern with the upcoming massive deployment of AI autonomy, and train a diverse, globally competitive workforce through education at all levels.
 

This research will create and validate new approaches for optimally managing mobile observational networks consisting of a renewably powered ?host? agent and ?satellite? agents that are deployed from and recharged by the host. Such networks can enable autonomous, long-term measurements for meteorological, climate change, reconnaissance, and surveillance applications, which are of significant national interest. While the hardware exists for such networks, the vast majority of existing mission planning and control approaches treat energy as a finite resource and focus on finite-duration missions. This research will represent a paradigm shift, wherein the energy resource available to the network is renewable, but the instantaneously available power is limited. This demands strategies that continuously trade off energy harvesting and scientific information gathering. This research will establish a comprehensive framework for managing the aforementioned tradeoffs, with both simulation-based and experimental demonstrations. The specific observational framework considered in this work will involve a fleet of solar-powered autonomous surface vessels, unoccupied aerial vehicles, and undersea gliders to for characterizing atmospheric and oceanic interactions between the deep-ocean and near-shore waters adjacent to North Carolina?s Outer Banks. The research will be complemented with targeted internship activities, K-12 outreach activities at The Engineering Place at NC State, and outreach activities with the Detroit Area Pre-College Engineering Program.

Fusing autonomy, persistence, and adaptation in observational networks demands a formal characterization and tradeoff between the cyber quantity of information and physical quantity of energy. Specifically, with a renewably powered host agent, energy no longer serves as a hard constraint; instead, there exists a perpetual tradeoff between the acquisition of information and the use of available on-board energy in a stochastic environment. To address this, the research team will create: (i) a scientifically tailored dynamic coverage model for information characterization, (ii) a statistical energy resource/consumption model, and (iii) a multi-level predictive controller that adapts the mission profile based on the information/energy tradeoff. The host controller will maximize a two-part objective function consisting of a finite-horizon coverage summation and terminal incentive based on a novel quantity termed the ?information value of energy.? This host controller will be complemented by a series of satellite energy-aware coverage controllers that maximize coverage subject to a safe rendezvous requirement in a stochastic resource. The research will be validated across three platforms of increasing complexity ? an unoccupied aerial vehicle (UAV) network (experimental), a combined solar-powered autonomous surface vessel (ASV)/UAV network (experimental), and a combined ASV/USV/undersea glider network (simulation-driven).

This Cyber-Physical Systems (CPS) project will make foundational methodological advances that enable safe and robust reinforcement learning (RL)-based control algorithmic solutions that are driven by problems in smart traffic signal control systems. Recent advances in computation, communication, storage, and sensing have led to a demand for data-driven learning-based decision-making and control in modern cyber-physical systems (CPSs), such as smart transportation systems. In such systems, decision-making agents need to operate safely and in a robust manner while working in complex environments with constraints that need to be respected. This project will develop foundational advances in robust RL solutions, and safe and constrained RL with provable guarantees by taking traffic signal control systems within smart transportation systems as our motivating CPS application and evaluation platform. This work will additionally focus on advancing curriculum development, recruitment of students from under-represented groups, involvement of undergraduate students in research, K-12 outreach, and also research community outreach via workshops, conference sessions, and seminars. The researchers will interface with companies and other stakeholders to communicate the results of the research as well as provide them with educational material on methodology. 

The technical approaches include: 1. Robust RL solutions incorporating model class knowledge, use of future predictions and robustness characterizations, and off-policy methods to address distributional shifts and data paucity arising from the use of a simulator/emulator or offline data; and 2. Efficient, safe, and constrained RL algorithms using model-free approaches and function-approximated methods, and also methods for partially-observed systems. To close the loop with the motivating CPS application, the RL algorithms will be evaluated in the context of traffic signal control via a comprehensive simulation-based evaluation using models of two instrumented sites.

Self-driving cars and home assistants provide just a small glimpse of the future cost-costrainted complex cyber-physical-human systems (CPHS) that will integrate engineering systems with the natural word and humans. This project will devise new mathematical tools and methods to systematically describe CPHS and optimize their operation. The application focus is on wireless body area networks, a natural CPHS representative with humans in the loop, heavily resource-constrained operation, and heterogeneous components that are intertwined with and altered by human behavior. The end will result will help understand important factors related to the operation of CPHS and how to optimize their operation. It will advance stochastic modeling, estimation and control theories to collectively address the challenges associated with this problem. It will also expose underrepresented K-12 students in Albany city to STEM fields from the lens of CPHS through project-based school visits and hands-on workshops on simple sensor systems, and prepare UAlbany students to become CPHS innovators by introducing CPHS concepts and activities in existing courses. Women and underrepresented groups will be encouraged to participate in this project by leveraging existing minority and underrepresented groups programs at the University at Albany.

CPHS have the potential to adaptively optimize their operation towards continuous real-time monitoring of an individual's state, environment and related behaviors, while providing real-time recommendations. To unleash the potential of CPHS, unique challenges related to sensing, communication, computation and control need to be jointly addressed in the presence of heterogeneous data, resource constraints and humans-in-the-loop. This fundamental research will advance cyber-physical systems science by devising new mathematical tools and methods, and a theoretical framework that can be used as a building block for various CPHS applications. Specifically, the project will (i) devise a new theoretical stochastic model to describe key CPHS variables and their interactions, (ii) design novel accurate and scalable estimators for CPHS and derive relevant theoretical performance bounds to quantify the fundamental limits of the estimation process in this context, and (iii) devise new controlled sensing, commmunication, and recommendations strategies to optimize the operation of systems with complex dynamics and heterogeneous capabilities.

This project will improve the ability to build artificial intelligence algorithms for Cyber-Physical Systems (CPS) that incorporate communications technologies by developing methods of learning from simulation environments. The specific application area is connected and automated vehicles (CAV) that drive strategically to reduce stop-and-go traffic. Employing communication between vehicles can improve the efficiency of vehicle control systems to manage traffic compared to vehicles without communication. The research of this project will explore the simulation of CAVs and how we can improve their algorithms to reduce traffic congestion, with core technology developments that are applicable to homes, health, and smart and connected communities. Increasingly at the heart of CPS are artificial intelligence algorithms, which can be programmed using a simulation of how the system should operate in the real world. A major challenge is building a simulation that accurately captures the complexity of the system in question, and how it can be controlled. The project includes partners from Toyota and Nissan that support testbeds enabling the research and accelerate transition of research to practice. The project aslo includes state and local Government stakeholders / partners which will facilitate experimentation in the real-world and demonstration of traffic congestion objectives as well as potentially emission reduction. Tools, technologies, and datasets generated in this project will be shared as active resources to support access beyond the life of the project. The project brings a focus on mentorship for undergraduate researchers, in order to broaden participation in computing. 

This project will develop new reinforcement learning approaches for Lagrangian control that accommodate communication and networking between actuators. A motivating domain that will be an application area of the project is CAVs. A major challenge is leveraging a small number of CAVs before those technologies realize full adoption rates. Vehicle and infrastructure communication technologies can be more useful for congestion management when feeding into a group of sparse, coordinated Lagrangian control agents. The project will use data from existing traffic sensors and testbeds to drive learning and control development. A fleet of instrumented and controllable passenger vehicles will be used for data collection and actuation. Validation experiments will be conducted using these vehicles on live roadways, and the results will be validated using a camera-based testbed that collects detailed traffic data.

This NSF project aims to develop a novel computational framework for informed contextual multi-armed bandits (iCMABs) that will be capable of robustly operating in complex, time-varying environments. The project will bring transformative change to the way that intelligent decision-making agents are designed for CPS, specifically those that utilize variants of multi-armed bandits. The intellectual merits of the project include: I) designing novel informed contextual bandits that maintain a generative model of their external world/environment, II) designing neural architecture search processed based on neuro-evolution to automatically design these generative models in an online manner, III) providing mechanisms to identify and address corrupt contextual and reward information, and IV) facilitating a process that enables the agent to generate predictions over longer-term horizons by querying its internal generative model. The broader impacts of the project include: I) advancing intelligent CPS through the iCMAB framework, II) providing decision-making modules and processes that readily integrate with many intelligent CPS/operations, and III) making important contributions to the field of machine learning and nature-inspired computing, specifically the automated design of intelligent agents based on artificial neural networks (ANNs). 

Current challenges faced by intelligent CPS, such as those used for tasks such as sensor validation and activity decisions, include being required to robustly operate in the face of noise, coming from things such as corrupted, fragmented, and uncertain reward values and context/state information, as well as having to adapt and make predictions in real-time and continually. Our proposed iCMAB framework will enable CPS to tackle these problems by: I) jointly evolving, in an online fashion, a reward forecasting model and a generative world model of contexts, II) providing a measure of confidence in predictions of both reward and context signals, and III) utilizing evolving recurrent neural networks (eRNNs) and brain-inspired neural systems/mechanisms to predict both contextual and reward information in the streaming data setting while mitigating catastrophic forgetting. This updated information will serve as input to CPS that are driven by contextual-bandits, enabling them to take more informed actions.