The objective of this proposal is to develop a distributed algorithmic framework, supported by a highly fault-tolerant software system, for executing critical transmission-level operations of the North American power grid using gigantic volumes of Synchrophasor data. As the number of Phasor Measurement Units (PMU) increases to more than thousands in the next 4-5 years, it is rather intuitive that the current state-of-the-art centralized communication and information processing architecture of Wide-Area Measurement System (WAMS) will no longer be sustainable under such data-explosion, and a completely distributed cyber-physical architecture will need to be developed. The North American Synchrophasor Initiative (NASPI) is currently addressing this architectural aspect by developing new communication and computing protocols through NASPI-net and Phasor Gateway. However, very little attention has been paid so far to perhaps the most critical consequence of this envisioned distributed architecture "namely", distributed algorithms, and their relevant middleware. Our primary task, therefore, will be to develop parallel computational methods for solving real-time wide-area monitoring and control problems with analytical investigation of their stability, convergence and robustness properties, followed by their implementation and testing against extraneous malicious attacks using our WAMS-RTDS testbed at NC State. In particular, we will address three critical research problems "namely" distributed wide-area oscillation monitoring, transient stability assessment, and voltage stability monitoring. The intellectual merit of this research will be in establishing an extremely timely application area of the PMU technology through its integration with distributed computing and optimal control. It will illustrate how ideas from advanced ideas from numerical methods and distributed optimization can be combined into power system monitoring and control applications, and how they can be implemented via fault-tolerant computing to maintain grid stability in face of catastrophic cyber and physical disturbances. The broader impact of this project will be in providing a much-needed application of CPS engineering to advance emerging research on PMU-integrated next-generation smart grids. Research results will be broadcast through journal publications, jointly organized graduate courses between NC State and University of Illinois Urbana Champagne, conference tutorials and workshops. Undergraduate research for minority engineering students will be promoted via the FREEDM Systems Center, summer internships via Information Trust Institute (UIUC) and RENCI, and middle/high-school student mentoring through the NCSU Science House program.

Cyber-Physical Systems (CPS) encompass a large variety of systems including for example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. One of the most important challenges in the design and deployment of Cyber-Physical Systems is how to formally guarantee that they are amenable to effective human control. This is a challenging problem not only because of the operational changes and increasing complexity of future CPS but also because of the nonlinear nature of the human-CPS system under realistic assumptions. Current state of the art has in general produced simplified models and has not fully considered realistic assumptions about system and environmental constraints or human cognitive abilities and limitations. To overcome current state of the art limitations, our overall research goal is to develop a theoretical framework for complex human-CPS that enables formal analysis and verification to ensure stability of the overall system operation as well as avoidance of unsafe operating states. To analyze a human-CPS involving a human operator(s) with bounded rationality three key questions are identified: (a) Are the inputs available to the operator sufficient to generate desirable behaviors for the CPS? (b) If so, how easy is it for the operator with her cognitive limitations to drive the system towards a desired behavior? (c) How can areas of poor system performance and determine appropriate mitigations be formally identified? The overall technical approach will be to (a) develop and appropriately leverage general cognitive models that incorporate human limitations and capabilities, (b) develop methods to abstract cognitive models to yield tractable analytical human models (c) develop innovative techniques to design the abstract interface between the human and underlying system to reflect mutual constraints, and (d) extend current state-of-the-art reachability and verification algorithms for analysis of abstract interfaces, iin which one of the systems in the feedback loop (i.e., the user) is mostly unknown, uncertain, highly variable or poorly modeled. The research will provide contributions with broad significance in the following areas: (1) fundamental principles and algorithms that would serve as a foundation for provably safe robust hybrid control systems for mixed human-CPS (2) methods for the development of analytical human models that incorporate cognitive abilities and limitations and their consequences in human control of CPS, (3) validated techniques for interface design that enables effective human situation awareness through an interface that ensures minimum information necessary for the human to safely control the CPS, (4) new reachability analysis techniques that are scalable and allow rapid determination of different levels of system safety. The research will help to identify problems (such as automation surprises, inadequate or excessive information contained in the user interface) in safety critical, high-risk, or expensive CPS before they are built, tested and deployed. The research will provide the formal foundations for understanding and developing human-CPS and will have a broad range of applications in the domains of healthcare, energy, air traffic control, transportation systems, homeland security and large-scale emergency response. The research will contribute to the advancement of under-represented students in STEM fields through educational innovation and outreach. The code, benchmarks and data will be released via the project website. Formal descriptions of models of human cognition are in general incompatible with formal models of the Cyber Physical System (CPS) the human operator(s) control. Therefore, it is difficult to determine in a rigorous way whether a CPS controlled by a human operator will be safe or stable and under which circumstances. The objective of this research is to develop an analytic framework of human-CPS systems that encompasses engineering compatible formal models of the human operator that preserve the basic architectural features of human cognition. In this project the team will develop methodologies for building such models as well as techniques for formal verification of the human-CPS system so that performance guarantees can be provided. They will validate models in a variety of domains ranging from air traffic control to large scale emergency response to the administration of anesthesia.

Design of cyber-physical systems today relies on executable models. Designers develop models, simulate them, find defects, and improve their designs before the system is built, thus greatly reducing the design costs. However, current model-based design methods lack support for model libraries (creating and exchanging models as "black boxes"), tool interoperability (allowing models to be co-simulated by multiple tools), and multi-view modeling (allowing to combine models that "live in different worlds", for instance, a control-logic model with an energy-consumption model). This project seeks to remedy this by developing a compositional modeling framework based on interfaces. Interfaces allow submodels to be treated as black boxes, exposing relevant information while hiding internal details. Success of the project will provide a solid theoretical foundation for compositionality in cyber-physical systems. Compositionality is a key property in system design, allowing to build systems in a scalable and modular manner. This project will enable the construction of model libraries, allowing the exchange of models developed by different teams, potentially coming from different disciplines and using different modeling languages and tools. Besides the considerable economic and societal impact of cyber-physical systems in general, the proposed project will have considerable impact on engineering and computer science education. Its focus on a rigorous and unified modeling theory will erode the boundaries between the currently separated cyber-physical system sub-disciplines that hamper competitiveness of our students. Finally, the project is strategically important for the competitiveness of the United States as it strengthens its presence in international standardization efforts for model exchange and co-simulation.

Accurate and reliable knowledge of time is fundamental to cyber-physical systems for sensing, control, performance, and energy efficient integration of computing and communications. This statement underlies the proposal. Emerging CPS applications depend on precise knowledge of time to infer location and control communication. There is a diversity of semantics used to describe time, and quality of time varies as we move up and down the system stack. System designs tend to overcompensate for these uncertainties and the result is systems that may be over designed, inefficient, and fragile. The intellectual merit derives from the new and fundamental concept of time and the holistic measure of quality of time (QoT) that captures metrics including resolution, accuracy, and stability. The proposal builds a system stack ("ROSELINE") that enables new ways for clock hardware, operating system, network services, and applications to learn, maintain and exchange information about time, influence component behavior, and robustly adapt to dynamic QoT requirements, as well as to benign and adversarial changes in operating conditions. Application areas that will benefit from Quality of Time will include: smart grad, networked and coordinated control of aerospace systems, underwater sensing, and industrial automation. The broader impact of the proposal is due to the foundational nature of the work which builds a robust and tunable quality of time that can be applied across a broad spectrum of applications that pervade modern life. The proposal will also provide valuable opportunities to integrate research and education in graduate, undergraduate, and K-12 classrooms. There will be extensive outreach through publications, open sourcing of software, and participation in activities such as the Los Angeles Computing Circle for pre-college students.

In telerobotic applications, human operators interact with robots through a computer network. This project is developing tools to prevent security threats in telerobotics, by monitoring and detecting malicious activities and correcting for them. To develop tools to prevent and mitigate security threats against telerobotic systems, this project adapts cybersecurity methods and extends them to cyber-physical systems. Knowledge about physical constraints and interactions between the cyber and physical components of the system are leveraged for security. A monitoring system is developed which collects operator commands and robot feedback information to perform real-time verification of the operator. Timely and reliable detection of any discrepancy between real and spoofed operator movements enables quick detection of adversarial activities. The results are evaluated on the UW-developed RAVEN surgical robot. This project brings together research in robotics, computer and network security, control theory and machine learning, in order to gain better understanding of complex teleoperated robotic systems and to engineer telerobotic systems that provide strict safety, security and privacy guarantees. The results are relevant and applicable to a wide range of applications, including telerobotic surgery, search and rescue missions, military operations, underwater infrastructure and repair, cleanup and repair in hazardous environments, mining, as well as manipulation/inspections of objects in low earth orbit. The project algorithms, software and hardware are being made available to the non-profit cyber-physical research community. Graduate and undergraduate students are being trained in cyber-physical systems security topics, and K-12, community college students and under-represented minority students are being engaged.

The objective of this proposal is to develop a distributed algorithmic framework, supported by a highly fault-tolerant software system, for executing critical transmission-level operations of the North American power grid using gigantic volumes of Synchrophasor data. As the number of Phasor Measurement Units (PMU) increases to more than thousands in the next 4-5 years, it is rather intuitive that the current state-of-the-art centralized communication and information processing architecture of Wide-Area Measurement System (WAMS) will no longer be sustainable under such data-explosion, and a completely distributed cyber-physical architecture will need to be developed. The North American Synchrophasor Initiative (NASPI) is currently addressing this architectural aspect by developing new communication and computing protocols through NASPI-net and Phasor Gateway. However, very little attention has been paid so far to perhaps the most critical consequence of this envisioned distributed architecture "namely", distributed algorithms, and their relevant middleware. Our primary task, therefore, will be to develop parallel computational methods for solving real-time wide-area monitoring and control problems with analytical investigation of their stability, convergence and robustness properties, followed by their implementation and testing against extraneous malicious attacks using our WAMS-RTDS testbed at NC State. In particular, we will address three critical research problems "namely" distributed wide-area oscillation monitoring, transient stability assessment, and voltage stability monitoring. The intellectual merit of this research will be in establishing an extremely timely application area of the PMU technology through its integration with distributed computing and optimal control. It will illustrate how ideas from advanced ideas from numerical methods and distributed optimization can be combined into power system monitoring and control applications, and how they can be implemented via fault-tolerant computing to maintain grid stability in face of catastrophic cyber and physical disturbances. The broader impact of this project will be in providing a much-needed application of CPS engineering to advance emerging research on PMU-integrated next-generation smart grids. Research results will be broadcast through journal publications, jointly organized graduate courses between NC State and University of Illinois Urbana Champagne, conference tutorials and workshops. Undergraduate research for minority engineering students will be promoted via the FREEDM Systems Center, summer internships via Information Trust Institute (UIUC) and RENCI, and middle/high-school student mentoring through the NCSU Science House program.

This project will result in fundamental physical and algorithmic building blocks of a novel cyber-physical for a two-way communication platform between handlers and working dogs designed to enable accurate training and control in open environments (eg, disaster response, emergency medical intervention). Miniaturized sensor packages will be developed to enable non- or minimally-invasive monitoring of dogs' positions and physiology. Activity recognition algorithms will be developed to blend data from multiple sensors. The algorithms will dynamically determine position and behavior from time series of inertial and physiological measurements. Using contextual information about task performance, the algorithms will provide duty-cycling information to reduce sensor power consumption while increasing sensing specificity. The resulting technologies will be a platform for implementation of communication. Strong interactions among computer science, electrical engineering, and veterinary science support this project. Work at the interface between electrical engineering and computer science will enable increased power efficiency and specificity of sensing in the detectors; work at the interface of electrical engineering and veterinary behavior will enable novel physiological sensing packages to be developed which measure behavioral signals in real time; Project outcomes will enable significant advances in how humans interact with both cyber and physical agents, including getting clearer pictures of behavior through real time physiological monitoring. Students are part of the project and multidisciplinary training will help to provide development of the Cyber-Physical Systems pipeline. Project outreach efforts will include working with middle school children, especially women and under-represented minorities, presentations in public museums that will promote public engagement and appreciation of the contribution of cyber-physical systems to daily lives. The goal of each outreach activity is to encourage both interest and excitement for STEM topics, demonstrating how computer science and engineering can lead to effective and engaging cyber-physical systems.

Processors in cyber-physical systems are increasingly being used in applications where they must operate in harsh ambient conditions and a computational workload which can lead to high chip temperatures. Examples include cars, robots, aircraft and spacecraft. High operating temperatures accelerate the aging of the chips, thus increasing transient and permanent failure rates. Current ways to deal with this mostly turn off the processor core or drastically slow it down when some part of it is seen to exceed a given temperature threshold. However, this pass/fail approach ignores the fact that (a) processors experience accelerated aging due to high temperatures, even if these are below the threshold, and (b) while deadlines are a constraint for real-time tasks to keep the controlled plant in the allowed state space, the actual controller response times that will increase if the voltage or frequency is lowered (to cool down the chip) are what determine the controlled plant performance. Existing approaches also fail to exploit the tradeoff between controller reliability (affected by its temperature history) and the performance of the plant. This project addresses these issues. Load-shaping algorithms are being devised to manage thermal stresses while ensuring appropriate levels of control quality. Such actions include task migration, changing execution speed, selecting an alternative algorithm or software implementation of control functions, and terminating prematurely optional portions of iterative tasks. Validation platforms for this project include automobiles and unmanned aerial vehicles. These platforms have been chosen based on both their importance to society and the significant technical challenges they pose. With CPS becoming ever more important in our lives and businesses, this project which will make CPS controllers more reliable and/or economical has broad potential social and economic impacts. Collaboration with General Motors promotes transition of the new technology to industry. The project includes activities to introduce students to thermal control in computing, in courses spanning high-school, undergraduate and graduate curricula.

The goal of the project is the development of the theory, hardware and computational infrastructure that will enable automatically transforming user-defined, high-level tasks such as inspection of hazardous environments and object retrieval, into provably-correct control for modular robots. Modular robots are composed of simple individual modules; while a single module has limited capabilities, connecting multiple modules in different configurations allows the system to perform complex actions such as climbing, manipulating objects, traveling in unstructured environments and self-reconfiguring (breaking into multiple independent robots and reassembling into larger structures). The project includes (i) defining and populating a large library of perception and actuation building blocks both manually through educational activities and automatically through novel algorithms, (ii) creating automated tools to assign values to probabilistic metrics associated with the performance of library components, (iii) developing a grammar and automated tools for control synthesis that sequence different components of the library to accomplish higher level tasks, if possible, or provide feedback to the user if the task cannot be accomplished and (iv) designing and building a novel modular robot platform capable of rapid and robust self-reconfiguration. This research will have several outcomes. First, it will lay the foundations for making modular robots easily controlled by anyone. This will enrich the robotic industry with new types of robots with unique capabilities. Second, the research will create novel algorithms that tightly combine perception, control and hardware capabilities. Finally, this project will create an open-source infrastructure that will allow the public to contribute basic controllers to the library thus promoting general research and social interest in robotics and engineering.

Objective: How much a person should be allowed to interact with a controlled machine? If that machine is safety critical, and if the computer that oversees its operation is essential to its operation and safety, the answer may be that the person should not be allowed to interfere with its operation at all or very little. Moreover, whether the person is a novice or an expert matters. Intellectual Merit: This research algorithmically resolves the tension between the need for safety and the need for performance, something a person may be much more adept at improving than a machine. Using a combination of techniques from numerical methods, systems theory, machine learning, human-machine interfaces, optimal control, and formal verification, this research will develop a computable notion of trust that allows the embedded system to assess the safety of the instruction a person is providing. The interface for interacting with a machine matters as well; designing motions for safety-critical systems using a keyboard may be unintuitive and lead to unsafe commands because of its limitations, while other interfaces may be more intuitive but threaten the stability of a system because the person does not understand the needs of the system. Hence, the person needs to develop trust with the machine over a period of time, and the last part of the research will include evaluating a person's performance by verifying the safety of the instructions the person provides. As the person becomes better at safe operation, she will be given more authority to control the machine while never putting the system in danger. Broader Impacts: The activities will include outreach, development of public-domain software, experimental coursework including two massive online courses, and technology transfer to rehabilitation. Outreach will include exhibits at the Museum of Science and Industry and working with an inner-city high school. The algorithms to be developed will have immediate impact on projects with the Rehabilitation Institute of Chicago, including assistive devices, stroke assessment, and neuromuscular hand control. Providing a foundation for a science of trust has the potential to transform rehabilitation research.