This NSF Cyber-Physical Systems (CPS) Frontier project "Verified Human Interfaces, Control, and Learning for Semi-Autonomous Systems (VeHICaL)" is developing the foundations of verified co-design of interfaces and control for human cyber-physical systems (h-CPS) --- cyber-physical systems that operate in concert with human operators. VeHICaL aims to bring a formal approach to designing both interfaces and control for h-CPS, with provable guarantees. The VeHICaL project is grounded in a novel problem formulation that elucidates the unique requirements on h-CPS including not only traditional correctness properties on autonomous controllers but also quantitative requirements on the logic governing switching or sharing of control between human operator and autonomous controller, the user interface, privacy properties, etc. The project is making contributions along four thrusts: (1) formalisms for modeling h-CPS; (2) computational techniques for learning, verification, and control of h-CPS; (3) design and validation of sensor and human-machine interfaces, and (4) empirical evaluation in the domain of semi-autonomous vehicles. The VeHICaL approach is bringing a conceptual shift of focus away from separately addressing the design of control systems and human-machine interaction and towards the joint co-design of human interfaces and control using common modeling formalisms and requirements on the entire system. This co-design approach is making novel intellectual contributions to the areas of formal methods, control theory, sensing and perception, cognitive science, and human-machine interfaces. Cyber-physical systems deployed in societal-scale applications almost always interact with humans. The foundational work being pursued in the VeHICaL project is being validated in two application domains: semi-autonomous ground vehicles that interact with human drivers, and semi-autonomous aerial vehicles (drones) that interact with human operators. A principled approach to h-CPS design --- one that obtains provable guarantees on system behavior with humans in the loop --- can have an enormous positive impact on the emerging national ``smart'' infrastructure. In addition, this project is pursuing a substantial educational and outreach program including: (i) integrating research into undergraduate and graduate coursework, especially capstone projects; (ii) extensive online course content leveraging existing work by the PIs; (iii) a strong undergraduate research program, and (iv) outreach and summer programs for school children with a focus on reaching under-represented groups.

Recent years have seen an explosion in the use of cellular and wifi networks to deploy fleets of semi-autonomous physical systems, including unmanned aerial vehicles (UAVs), self-driving vehicles, and weather stations to perform tasks such as package delivery, crop harvesting, and weather prediction. The use of cellular and wifi networks has dramatically decreased the cost, energy, and maintenance associated with these forms of embedded technology, but has also added new challenges in the form of delay, packet drops, and loss of signal. Because of these new challenges, and because of our limited understanding of how unreliable communication affects performance, the current protocols for regulating physical systems over wireless networks are slow, inefficient, and potentially unstable. In this project we develop a new computational framework for designing provably fast, efficient and safe protocols for the control of fleets of semi-autonomous physical systems. The systems considered in this project are dynamic, defined by coupled ordinary differential equations, and connected by feedback to a controller, with a feedback interconnection which has multiple static delays, multiple time-varying delays, or is sampled at discrete times. For these systems, we would like to design optimal and robust feedback controllers assuming a limited number of sensor measurements are available. Specifically, we seek to design a class of algorithms which are computationally efficient, which scale to large numbers of subsystems, and which, given models of the dynamics, communication links, and uncertainty, will return a controller which is provably stable, robust to model uncertainty, and provably optimal in the relevant metric of performance. To accomplish this task, we leverage a new duality result which allows the problem of controller synthesis for infinite-dimensional systems to be convexified. This result allows the problem of optimal and robust dynamic output-feedback controller synthesis to be reformulated as feasibility of a set of convex linear operator inequalities. We then use semidefinite programming to parametrize the set of feasible operators and thereby test feasibility of the inequalities with little to no conservatism. In a similar manner, estimator design and optimal controller synthesis are recast as semidefinite programming problems and used to solve the problems of sampled-data and systems with input delay. The algorithms will be scalable to at least 20 states and the controllers will be field-tested on a fleet of wheeled robotic vehicles.

Coordinated cyber-physical attacks (CCPA) have been touted as a serious threat for several years, where "coordinated" means that attackers have complete knowledge of the physical plant and status, and sometimes can even create physical defects, to assist cyber attacks, and vice versa. In recent years, these attacks have crept from theory to reality, with attacks on vehicles, electrical grids, and industrial plants, which have the potential to cause destruction and even death outside of the digital world. CCPA raise a unique challenge with respect to cyber-physical systems (CPS) safety. Historically, technologies to defend cyber attacks and physical attacks are developed separately under different assumptions and models. For instance, cyber security technologies often require the complete profile of the physical dynamics and the observation of the system state, which may not be available when physical defects exist. Similarly, existing system control techniques may efficiently compensate for the physical damage, but under the assumption that the control software and the sensor data are not compromised. There is a lack of unified approaches against CCPA. With this observation, this project focuses on the development of unified models with coherent set of assumptions, supported by integrated technologies, upon which CCPA can be defended much more effectively. To establish theoretical foundations and engineering principles for resilient CPS architectures, this project will investigate unified models and platforms that represent the scientific understanding of resilient CPS against CCPA. Engineering of CPS will be addressed through the development and integration of complexity-reduced software architectures, along with their design principles, which lead to verifiable and certifiable architectures with higher level of system resilience. Technology of CPS will be addressed through the design of new attack detection, isolation, and recovery tools as well as timing and control techniques to ensure appropriate responses to CCPA. The proposed inherently interdisciplinary research will ensure predictable performance for resilient CPS, by leveraging the disciplinary advances in (i) the design and evaluation of robust fault-tolerant control systems yielding significantly enhanced levels of safety in highly unpredictable environments; (ii) the design and implementation of complexity reduction architecture yielding a significant reduction in the verification time from hours to seconds; (iii) the development of multi-rate sampled-data control and robust reachability-based attack detection techniques ensuring that the sensor data is reliable; and (iv) the development of cyber-physical co-adaptation that optimizes control performance and computation task scheduling to guarantee system safety and efficient recovery from CCPA. The target application of this project is unmanned aerial vehicles (UAVs). The research results will be evaluated in three different testbeds: UAV testbed, generic transportation model (GTM) aircraft, and power system virtual testbed (VTB). The technological advancement from this project will provide solutions for the safety and reliability issues faced by today's CPS and deliver dependable CPS that are applicable without sacrificing functionality or accessibility in complex and potentially hostile networked environment. The results of this project will be communicated in archival journal publications, conference venues and various workshops and lectures, and will be integrated at different academic levels.

The goal of this project is to investigate a low-cost and energy-efficient hardware and software system to close the loop between processing of sensor data, semantically high-level detection and trajectory generation in real-time. To safely integrate Unmanned Aerial Vehicles into national airspace, there is an urgent need to develop onboard sense-and-avoid capability. While deep neural networks (DNNs) have significantly improved the accuracy of object detection and decision making, they have prohibitively high complexity to be implemented on small UAVs. Moreover, existing UAV flight control approaches ignore the nonlinearities of UAVs and do not provide trajectory assurance. The research thrusts of this project are: (i) FPGA implementation of DNNs: both fully connected and convolutional layers of deep (convolutional) neural networks will be trained using (block-)circulant matrix and implemented using custom designed universal Fast Fourier Transform kernels on FPGA. This research thrust will enable efficient implementation of DNNs, reducing memory and computation complexity from O(N2) to O(N) and O(NlogN), respectively; (ii) autonomous detection and perception for onboard sense-and-avoid: existing regional detection neural networks will be extended to work with images taken from different angles, and multi-modal sensor inputs; (iii) real-time waypoint and trajectory generation - an integrated trajectory generation and feedback control scheme for steering under-actuated vehicles through desired waypoints in 3D space will be developed. For efficient implementation and hardware reuse, both detection and control problems will be formulated and solved using DNNs with (block-)circulant weight matrix. Deep reinforcement learning models will be investigated for waypoint generation and to assign artificial potential around the obstacles to guarantee a safe distance. The fundamental research results will enable onboard computing, real-time detection and control, which are cornerstones of autonomous and next-generation UAVs.

This proposal is for research on the Mobile Automated Rovers Fly-By (MARS-FLY) for Bridge Network Resiliency. Bridges are often in remote locations and the cost of installing electricity and a data acquisition system in hundreds of thousands of bridges is prohibitive. The MARS-FLY project will develop a cyber-physical system (CPS) designed to monitor the health of highway bridges, control the loads imposed on bridges by heavy trucks, and provide visual inspectors with quantitative information for data-driven bridge health assessment requiring no electricity and a minimum of data acquisition electronics on site. For fly-by monitoring, GPS-controlled auto-piloted drones will periodically carry data acquisition electronics to the bridge and download the data from the sensors at a close range. Larger Imaging drones carrying infrared (IR) cameras will be used to detect detail damages like concrete delamination. The research objectives will be accomplished first, by wireless recharging of remote sensor motes by drone to enhance the sensor operational lifetime whereas wireless recharging of drone battery will extend the operational efficiency, payload, and drone range. The novel multi-coil wireless powering approach will provide an investigation of an engineered material i.e. metamaterial with the resonant link to enhance the power level and link distance, otherwise unachievable. Next, by a major scientific breakthrough in the utilization of small quantities of low quality sensor data and IR images to determine damage information at all levels: detection of a change in behavior, location, and magnitude; streamlining of reliability analysis to incorporate the new information of damage into the bridge's reliability index based on combined numerical and probabilistic approaches such as Ensemble Empirical Mode Decomposition with the Hilbert Transform; and finally detection of nonlinearities in the signals in a Bayesian Updating framework. Moreover, an instrumented drive-by vehicle will complement damage detection on the bridge. A Bayesian updating framework will be used to update the probability distribution for bridge condition, given the measurements. Image processing of the infrared images to distinguish between the environmental effects and the true bridge deterioration (e.g. delamination in concrete) will be used to develop a better method of site-specific and environment-specific calibration.