This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.

This project investigates new reinforcement learning algorithms to enable long-term real-time autonomous learning by cyber-physical systems (CPS). The complexity of CPS makes hand-programming safe and efficient controllers for them difficult. For CPS to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes with potential for solving this problem. However, existing RL algorithms do not meet all of the requirements of learning in CPS. Efficacy of the new algorithms for CPS is evaluated in the context of smart buildings and autonomous vehicles. Cyber-physical systems (CPS) have the potential to revolutionize society by enabling smart buildings, transportation, medical technology, and electric grids. Success of this project could lead to a new generation of CPS that are capable of adapting to their situation and improving their performance autonomously over time. In addition to the traditional methods of dissemination, this project will develop and release open-source code implementing the new reinforcement learning algorithms. Education and outreach activities associated with the project include a Freshman Research Initiative course, participation in a UT Austin annual open house that draws in many underrepresented minorities to interest the public in computer science and science in general, and the department's annual summer school for high school girls called First Bytes.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.

Reliable operation of cyber-physical systems (CPS) of societal importance such as Smart Electric Grids is critical for the seamless functioning of a vibrant economy. Sustained power outages can lead to major disruptions over large areas costing millions of dollars. Efficient computational techniques and tools that curtail such systematic failures by performing fault diagnosis and prognostics are therefore necessary. The Smart Electric Grid is a CPS: it consists of networks of physical components (including generation, transmission, and distribution facilities) interfaced with cyber components (such as intelligent sensors, communication networks, and control software). This grant provides funding to develop new methods to build models for the smart grid representing the failure dependencies in the physical and cyber components. The models will be used to build an integrated system-wide solution for diagnosing faults and predicting future failure propagations that can account for existing protection mechanisms. The original contribution of this work will be in the integrated modeling of failures on multiple levels in a large distributed cyber-physical system and the development of novel, hierarchical, robust, online algorithms for diagnostics and prognostics. If successful, the model-based fault diagnostics and prognostics techniques will improve the effectiveness of isolating failures in large systems by identifying impending failure propagations and determining the time to critical failures that will increase system reliability and reduce the losses accrued due to failures. This work will bridge the gap between fault management approaches used in computer science and power engineering that are needed as the grid becomes smarter, more complex, and more data intensive. Outcomes of this project will include modeling and run-time software prototypes, research publications, and experimental results in collaborations with industry partners that will be made available to the scientific community.

Reliable operation of cyber-physical systems (CPS) of societal importance such as Smart Electric Grids is critical for the seamless functioning of a vibrant economy. Sustained power outages can lead to major disruptions over large areas costing millions of dollars. Efficient computational techniques and tools that curtail such systematic failures by performing fault diagnosis and prognostics are therefore necessary. The Smart Electric Grid is a CPS: it consists of networks of physical components (including generation, transmission, and distribution facilities) interfaced with cyber components (such as intelligent sensors, communication networks, and control software). This grant provides funding to develop new methods to build models for the smart grid representing the failure dependencies in the physical and cyber components. The models will be used to build an integrated system-wide solution for diagnosing faults and predicting future failure propagations that can account for existing protection mechanisms. The original contribution of this work will be in the integrated modeling of failures on multiple levels in a large distributed cyber-physical system and the development of novel, hierarchical, robust, online algorithms for diagnostics and prognostics. If successful, the model-based fault diagnostics and prognostics techniques will improve the effectiveness of isolating failures in large systems by identifying impending failure propagations and determining the time to critical failures that will increase system reliability and reduce the losses accrued due to failures. This work will bridge the gap between fault management approaches used in computer science and power engineering that are needed as the grid becomes smarter, more complex, and more data intensive. Outcomes of this project will include modeling and run-time software prototypes, research publications, and experimental results in collaborations with industry partners that will be made available to the scientific community.

Accurate and reliable knowledge of time is fundamental to cyber-physical systems for sensing, control, performance, and energy efficient integration of computing and communications. This statement underlies the proposal. Emerging CPS applications depend on precise knowledge of time to infer location and control communication. There is a diversity of semantics used to describe time, and quality of time varies as we move up and down the system stack. System designs tend to overcompensate for these uncertainties and the result is systems that may be over designed, inefficient, and fragile. The intellectual merit derives from the new and fundamental concept of time and the holistic measure of quality of time (QoT) that captures metrics including resolution, accuracy, and stability. The proposal builds a system stack ("ROSELINE") that enables new ways for clock hardware, operating system, network services, and applications to learn, maintain and exchange information about time, influence component behavior, and robustly adapt to dynamic QoT requirements, as well as to benign and adversarial changes in operating conditions. Application areas that will benefit from Quality of Time will include: smart grad, networked and coordinated control of aerospace systems, underwater sensing, and industrial automation. The broader impact of the proposal is due to the foundational nature of the work which builds a robust and tunable quality of time that can be applied across a broad spectrum of applications that pervade modern life. The proposal will also provide valuable opportunities to integrate research and education in graduate, undergraduate, and K-12 classrooms. There will be extensive outreach through publications, open sourcing of software, and participation in activities such as the Los Angeles Computing Circle for pre-college students.

The objective of this proposal is to develop a distributed algorithmic framework, supported by a highly fault-tolerant software system, for executing critical transmission-level operations of the North American power grid using gigantic volumes of Synchrophasor data. As the number of Phasor Measurement Units (PMU) increases to more than thousands in the next 4-5 years, it is rather intuitive that the current state-of-the-art centralized communication and information processing architecture of Wide-Area Measurement System (WAMS) will no longer be sustainable under such data-explosion, and a completely distributed cyber-physical architecture will need to be developed. The North American Synchrophasor Initiative (NASPI) is currently addressing this architectural aspect by developing new communication and computing protocols through NASPI-net and Phasor Gateway. However, very little attention has been paid so far to perhaps the most critical consequence of this envisioned distributed architecture "namely", distributed algorithms, and their relevant middleware. Our primary task, therefore, will be to develop parallel computational methods for solving real-time wide-area monitoring and control problems with analytical investigation of their stability, convergence and robustness properties, followed by their implementation and testing against extraneous malicious attacks using our WAMS-RTDS testbed at NC State. In particular, we will address three critical research problems "namely" distributed wide-area oscillation monitoring, transient stability assessment, and voltage stability monitoring. The intellectual merit of this research will be in establishing an extremely timely application area of the PMU technology through its integration with distributed computing and optimal control. It will illustrate how ideas from advanced ideas from numerical methods and distributed optimization can be combined into power system monitoring and control applications, and how they can be implemented via fault-tolerant computing to maintain grid stability in face of catastrophic cyber and physical disturbances. The broader impact of this project will be in providing a much-needed application of CPS engineering to advance emerging research on PMU-integrated next-generation smart grids. Research results will be broadcast through journal publications, jointly organized graduate courses between NC State and University of Illinois Urbana Champagne, conference tutorials and workshops. Undergraduate research for minority engineering students will be promoted via the FREEDM Systems Center, summer internships via Information Trust Institute (UIUC) and RENCI, and middle/high-school student mentoring through the NCSU Science House program.

Accurate and reliable knowledge of time is fundamental to cyber-physical systems for sensing, control, performance, and energy efficient integration of computing and communications. This statement underlies the proposal. Emerging CPS applications depend on precise knowledge of time to infer location and control communication. There is a diversity of semantics used to describe time, and quality of time varies as we move up and down the system stack. System designs tend to overcompensate for these uncertainties and the result is systems that may be over designed, inefficient, and fragile. The intellectual merit derives from the new and fundamental concept of time and the holistic measure of quality of time (QoT) that captures metrics including resolution, accuracy, and stability. The proposal builds a system stack ("ROSELINE") that enables new ways for clock hardware, operating system, network services, and applications to learn, maintain and exchange information about time, influence component behavior, and robustly adapt to dynamic QoT requirements, as well as to benign and adversarial changes in operating conditions. Application areas that will benefit from Quality of Time will include: smart grad, networked and coordinated control of aerospace systems, underwater sensing, and industrial automation. The broader impact of the proposal is due to the foundational nature of the work which builds a robust and tunable quality of time that can be applied across a broad spectrum of applications that pervade modern life. The proposal will also provide valuable opportunities to integrate research and education in graduate, undergraduate, and K-12 classrooms. There will be extensive outreach through publications, open sourcing of software, and participation in activities such as the Los Angeles Computing Circle for pre-college students.

Accurate and reliable knowledge of time is fundamental to cyber-physical systems for sensing, control, performance, and energy efficient integration of computing and communications. This statement underlies the proposal. Emerging CPS applications depend on precise knowledge of time to infer location and control communication. There is a diversity of semantics used to describe time, and quality of time varies as we move up and down the system stack. System designs tend to overcompensate for these uncertainties and the result is systems that may be over designed, inefficient, and fragile. The intellectual merit derives from the new and fundamental concept of time and the holistic measure of quality of time (QoT) that captures metrics including resolution, accuracy, and stability. The proposal builds a system stack ("ROSELINE") that enables new ways for clock hardware, operating system, network services, and applications to learn, maintain and exchange information about time, influence component behavior, and robustly adapt to dynamic QoT requirements, as well as to benign and adversarial changes in operating conditions. Application areas that will benefit from Quality of Time will include: smart grad, networked and coordinated control of aerospace systems, underwater sensing, and industrial automation. The broader impact of the proposal is due to the foundational nature of the work which builds a robust and tunable quality of time that can be applied across a broad spectrum of applications that pervade modern life. The proposal will also provide valuable opportunities to integrate research and education in graduate, undergraduate, and K-12 classrooms. There will be extensive outreach through publications, open sourcing of software, and participation in activities such as the Los Angeles Computing Circle for pre-college students.

The objective of this project is to research tools to manage uncertainty in the design and certification process of safety-critical aviation systems. The research focuses on three innovative ideas to support this objective. First, probabilistic techniques will be introduced to specify system-level requirements and bound the performance of dynamical components. These will reduce the design costs associated with complex aviation systems consisting of tightly integrated components produced by many independent engineering organizations. Second, a framework will be created for developing software components that use probabilistic execution to model and manage the risk of software failure. These techniques will make software more robust, lower the cost of validating code changes, and allow software quality to be integrated smoothly into overall system-level analysis. Third, techniques from Extreme Value Theory will be applied to develop adaptive verification and validation procedures. This will enable early introduction of new and advanced aviation systems. These systems will initially have restricted capabilities, but these restrictions will be gradually relaxed as justified by continual logging of data from in-service products. The three main research aims will lead to a significant reduction in the costs and time required for fielding new aviation systems. This will enable, for example, the safe and rapid implementation of next generation air traffic control systems that have the potential of tripling airspace capacity with no reduction in safety. The proposed methods are also applicable to other complex systems including smart power grids and automated highways. Integrated into the research is an education plan for developing a highly skilled workforce capable of designing safety critical systems. This plan centers around two main activities: (a) creation of undergraduate labs focusing on safety-critical systems, and (b) integration of safety-critical concepts into a national robotic snowplow competition. These activities will provide inspirational, real-world applications to motivate student learning.