The electric power grid experiences disturbances all the time that are routinely controlled, managed, or eliminated by system protection measures- designed by careful engineering studies and fine-tuned by condensing years of operational experience. Despite this, the grid sometimes experiences disruptive events that can quickly, and somewhat unstoppably catapult the system towards a blackout. Arresting such blackouts has remained elusive - mainly because relays (protection devices) operate on local data, and are prone to hidden faults that are impossible to detect until they manifest, resulting in misoperations that have sometime been precipitators or contributors to blackouts. Inspired by the Presidential policy directive on resilience -- meaning the ability to anticipate, prepare, withstand, and recover from disruptive events, this project proposes "WARP: Wide Area assisted Resilient Protection", a paradigm that adds a layer of finer (supervisory) intelligence to supplement conventional protection wisdom - which we call resilient protection. Exploiting high fidelity measurements and computation to calculate and analyze energy function components of power systems to identify disturbances, WARP would allow relays to be supervised - correct operations would be corroborated, and misoperations will be remedied by judiciously reversing the relay operation in a rational time-frame. The project also envisions predicting instability using advanced estimation techniques, thus being proactive. This will provide power grid the ability to auto-correct and bounce back from misoperations, curtailing the size, scale and progression of blackouts and improving the robustness and resilience of the electric grid -- our nation's most critical infrastructure. In WARP, disruptive events are deciphered by using synchrophasor data, energy functions, and dynamic state information via particle filtering. The information is fused to provide a global data set and intelligence signal that supervises relays, and also to predict system stability. Resilience is achieved when the supervisory signal rectifies the misoperation of relays, or endorses their action when valid. This endows relays with post-event-auto-correct abilities 
- a feature that never been explored/understood in the protection-stability nexus. Architectures to study the effect of latency and bad data are proposed. WARP introduces new notions: global detectability and distinguishability for power system events, stability prediction based on
the sensitivity of the energy function components and uses a novel factorization method: (CUR) preserving data interpretability to reduce data dimensionality. All the proposed
tools will be wrapped into a simulation framework to assess scalability and accuracy-runtime tradeoffs, and quantify the degree of resilience achieved. The effectiveness of the proposed scheme during extreme events will be measured by reenacting two well-documented blackout sequences. In addition, simulations on benchmarked systems will be performed to assess scalability and accuracy-runtime tradeoffs, and quantify the degree of resilience achieved.

Power systems have seen many changes over the last decade including the increased penetration of renewable generation, electric vehicles and new technologies for sensing, communication and control of a Smart Grid. The most significant impact of these changes are being felt at the consumer level. The ability for consumers and end devices to buy and sell energy and related services in a dynamic and interactive manner is expected to create a transactive energy market as highlighted in the Dec 2014 report of GridWise Alliance. Modeling and preparing the physical system to respond to the somewhat unpredictable behavior of active consumers over a cyber-infrastructure will be critical for maintaining grid reliability. Understanding the impact of such active consumers on the operational and business policies of the distribution utility requires advances in core system science that spans the areas of power engineering, economics, statistical signal processing, game theory, distributed control, multi-agent systems and cyber security. In conjunction with industrial partners, Westar Energy (the largest electric company in Kansas) and Kansas City Power and Light, the PIs plan to develop an architecture that requires little change to the existing investment in power distribution systems while allowing for the dynamic, adaptive control required to integrate active consumers with current and future combinations of high-variability distributed power sources, such as Photo-voltaic (PV) generators and storage batteries. In contrast to prior related efforts that primarily focus on demand response and distributed generation management with a single home/user centric approach, the proposed approach takes a holistic system perspective that includes cumulative modeling of multiple stochastic active consumers and the cyber infrastructure over which they may interact. Specific research thrusts include: (1) a general, extensible, and secure cyber architecture based on holonic multi-agent principles that provides a pathway to the emerging area of transactive energy market in power distribution systems, but also provides foundation for other engineered systems with active consumers; (2) new analytical insights into generalized stochastic modeling of consumer response to real]time price of electricity and the impact of such active consumers on grid reliability and security, and (3) novel methodology for comprehensive distributed control and management of power distribution systems with active consumers and high penetration of distributed renewable resources. Active consumers are an integral part of the Smart City vision where cyber systems are integrated into the transportation, energy, healthcare and biomedical, and critical infrastructure systems. Successful completion of this project will result in modeling, control, analysis and simulation architectures for all such active consumer driven CPS domains. The resulting gains in operating efficiency, economics, reliability and security will result in overall welfare for the society.

Smart grid includes two interdependent infrastructures: power transmission and distribution network, and the supporting telecommunications network. Complex interactions among these infrastructures lead to new pathways for attack and failure propagation that are currently not well understood. This innovative project takes a holistic multilevel approach to understand and characterize the interdependencies between these two infrastructures, and devise mechanisms to enhance their robustness. Specifically, the project has four goals. The first goal is to understand the standardized smart grid communications protocols in depth and examine mechanisms to harden them. This is essential since the current protocols are notoriously easy to attack. The second goal is to ensure robustness in state estimation techniques since they form the basis for much of the analysis of smart grid. In particular, the project shall exploit a steganography-based approach to detect bad data and compromised devices. The third goal is to explore trust-based attack detection strategies that combine the secure state estimation with power flow models and software attestation to detect and isolate compromised components. The final goal is to study reconfiguration strategies that combine light-weight prediction models, stochastic decision processes, intentional islanding, and game theory techniques to mitigate the spreading of failures and the loss of load. A unique aspect of smart grid security that will be studied in this project is the critical importance of timeliness, and thus a tradeoff between effectiveness of the mechanisms and the overhead introduced. The project is expected to provide practical techniques for making the smart grid more robust against failures and attacks, and enable it to recover from large scale failures with less loss of capacity. The project will also train students in the multidisciplinary areas of power systems operation and design, networking protocols, and cyber-physical security.

The wide-area measurement systems technology using Phasor Measurement Units (PMUs) has been regarded as the key to guaranteeing stability, reliability, state estimation, and control of next-generation power systems. However, with the exponentially increasing number of PMUs, and the resulting explosion in data volume, the design and deployment of an efficient wide-area communication and computing infrastructure is evolving as one of the greatest challenges to the power system and IT communities. The goal of this NSF CPS project is to address this challenge, and construct a massively deployable cyber-physical architecture for wide-area control that is fast, resilient and cost-optimal (FRESCO). The FRESCO grid will consist of a suite of optimal control algorithms for damping oscillations in power flows and voltages, implemented on top of a cost-effective and cyber-secure distributed computing infrastructure connected by high-speed wide-area networks that are dynamically programmable and reconfigurable. The value of constructing FRESCO is twofold (1) If a US-wide communication network capable of transporting gigabit volumes of PMU data for wide-area control indeed needs to be implemented over the next five years then power system operators must have a clear sense of how various forms of delays, packet losses, and security threats affect the stability of these control loops. (2) Moreover, such wide-area communication must be made economically feasible and sustainable via joint decision-making processes between participating utility companies, and testing how controls can play a potential role in facilitating such economics. Currently, there is very limited insight into how the PMU data transport protocols may lead to a variety of such delay patterns, or dictate the economic investments. FRESCO will answer all of these questions, starting from small prototypical grid models to those with tens of thousands of buses. Our eventual goal will be to make FRESCO fully open-source for Transition to Practice (TTP). We will work with two local software companies in Raleigh, namely Green Energy Corporation and Real-Time Innovations, Inc. to develop a scalable, secure middleware using Data-Distribution Service (DDS) technology. Thus, within the scope of the project, we also expect to enrich the state-of-the-art cloud computing and networking technologies with new control and management functions. From a technical perspective, FRESCO will answer three main research questions. First, can wide-area controllers be co-designed in sync with communication delays to make the closed-loop system resilient and delay-aware, rather than just delay-tolerant This is particularly important, as PMU data, in most practical scenarios, will have to be transported over a shared resource, sharing bandwidth with other ongoing applications, giving rise to not only transport delays, but also significant delays due to queuing and routing. Advanced ideas of arbitrated network control designs will be used to address this problem. The second question we address is for cost. Given that there are several participants in this wide-area control, how much is each participant willing to pay in sharing the network cost with others for the sake of supporting a system-wide control objective compared to its current practice of opting for selfish feedback control only Ideas from cooperative game theory will be used to investigate this problem. The final question addresses security how can one develop a scientific methodology to assess risks, and mitigate security attacks in wide-area control? Statistical and structural analysis of attack defense modes using Bayesian and Markov models, game theory, and discrete-event simulation will be used to address this issue. Experimental demos will be carried out using the DETER-WAMS network, showcasing the importance of cyber-innovation for the sustainability of energy infrastructures. Research results will be broadcast through journal publications, and jointly organized graduate courses between NCSU, MIT and USC.

As information technology has transformed physical systems such as the power grid, the interface between these systems and their human users has become both richer and much more complex. For example, from the perspective of an electricity consumer, a whole host of devices and technologies are transforming how they interact with the grid: demand response programs; electric vehicles; "smart" thermostats and appliances; etc. These novel technologies are also forcing us to rethink how the grid interacts with its users, because critical objectives such as stability and robustness require effective integration among the many diverse users in the grid. This project studies the complex interweaving of humans and physical systems. Traditionally, a separation principle has been used to isolate humans from physical systems. This principle requires users to have preferences that are well-defined, stable, and quickly discoverable. These assumptions are increasingly violated in practice: users' preferences are often not well-defined; unstable over time; and take time to discover. Our project articulates a new framework for interactions between physical systems and their users, where users' preferences must be repeatedly learned over time while the system continually operates with respect to imperfect preference information. We focus on the area of power systems. Our project has three main thrusts. First, user models are rethought to reflect the fact this new dynamic view of user preferences, where even the users are learning over time. The second thrust focuses on developing a new system model that learns about users, since we cannot understand users in a "single-shot"; rather, repeated interaction with the user is required. We then focus on the integration of these two new models. How do we control and operate a physical system, in the presence of the interacting "learning loops", while mediating between many competing users? We apply ideas from mean field games and optimal power flow to capture, analyze, and transform the interaction between the system and the ongoing preference discovery process. Our methods will yield guidance for market design in power systems where user preferences are constantly evolving. If successful, our project will usher in a fundamental change in interfacing physical systems and users. For example, in the power grid, our project directly impacts how utilities design demand response programs; how smart devices learn from users; and how the smart grid operates. In support of this goal, the PIs intend to develop avenues for knowledge transfer through interactions with industry. The PIs will also change their education programs to reflect a greater entanglement between physical systems and users.

Inadequate system understanding and inadequate situational awareness have caused large-scale power outages in the past. With the increased reliance on variable energy supply sources, system understanding and situational awareness of a complex energy system become more challenging. This project leverages the power of big data analytics to directly improve system understanding and situational awareness. The research provides the methodology for detecting anomalous events in real-time, and therefore allow control centers to take appropriate control actions before minor events develop into major blackouts. The significance for the society and for the power industry is profound. Energy providers will be able to prevent large-scale power outages and reduce revenue losses, and customers will benefit from reliable energy delivery with service guarantees. Students, including women and underrepresented groups, will be trained for the future workforce in this area. The project includes four major thrusts: 1) real-time anomaly detection from measurement data; 2) real-time event diagnosis and interpretation of changes in the state of the network; 3) real-time optimal control of the power grid; 4) scientific foundations underpinning cyber-physical systems. The major outcome of this project is practical solutions to event or fault detection and diagnosis in the power grid, as well as prediction and prevention of large-scale power outages.

The wide-area measurement systems technology using Phasor Measurement Units (PMUs) has been regarded as the key to guaranteeing stability, reliability, state estimation, and control of next-generation power systems. However, with the exponentially increasing number of PMUs, and the resulting explosion in data volume, the design and deployment of an efficient wide-area communication and computing infrastructure is evolving as one of the greatest challenges to the power system and IT communities. The goal of this NSF CPS project is to address this challenge, and construct a massively deployable cyber-physical architecture for wide-area control that is fast, resilient and cost-optimal (FRESCO). The FRESCO grid will consist of a suite of optimal control algorithms for damping oscillations in power flows and voltages, implemented on top of a cost-effective and cyber-secure distributed computing infrastructure connected by high-speed wide-area networks that are dynamically programmable and reconfigurable. The value of constructing FRESCO is twofold (1) If a US-wide communication network capable of transporting gigabit volumes of PMU data for wide-area control indeed needs to be implemented over the next five years then power system operators must have a clear sense of how various forms of delays, packet losses, and security threats affect the stability of these control loops. (2) Moreover, such wide-area communication must be made economically feasible and sustainable via joint decision-making processes between participating utility companies, and testing how controls can play a potential role in facilitating such economics. Currently, there is very limited insight into how the PMU data transport protocols may lead to a variety of such delay patterns, or dictate the economic investments. FRESCO will answer all of these questions, starting from small prototypical grid models to those with tens of thousands of buses. Our eventual goal will be to make FRESCO fully open-source for Transition to Practice (TTP). We will work with two local software companies in Raleigh, namely Green Energy Corporation and Real-Time Innovations, Inc. to develop a scalable, secure middleware using Data-Distribution Service (DDS) technology. Thus, within the scope of the project, we also expect to enrich the state-of-the-art cloud computing and networking technologies with new control and management functions. From a technical perspective, FRESCO will answer three main research questions. First, can wide-area controllers be co-designed in sync with communication delays to make the closed-loop system resilient and delay-aware, rather than just delay-tolerant This is particularly important, as PMU data, in most practical scenarios, will have to be transported over a shared resource, sharing bandwidth with other ongoing applications, giving rise to not only transport delays, but also significant delays due to queuing and routing. Advanced ideas of arbitrated network control designs will be used to address this problem. The second question we address is for cost. Given that there are several participants in this wide-area control, how much is each participant willing to pay in sharing the network cost with others for the sake of supporting a system-wide control objective compared to its current practice of opting for selfish feedback control only Ideas from cooperative game theory will be used to investigate this problem. The final question addresses security how can one develop a scientific methodology to assess risks, and mitigate security attacks in wide-area control? Statistical and structural analysis of attack defense modes using Bayesian and Markov models, game theory, and discrete-event simulation will be used to address this issue. Experimental demos will be carried out using the DETER-WAMS network, showcasing the importance of cyber-innovation for the sustainability of energy infrastructures. Research results will be broadcast through journal publications, and jointly organized graduate courses between NCSU, MIT and USC.

The electric power grid experiences disturbances all the time that are routinely controlled, managed, or eliminated by system protection measures- designed by careful engineering studies and fine-tuned by condensing years of operational experience. Despite this, the grid sometimes experiences disruptive events that can quickly, and somewhat unstoppably catapult the system towards a blackout. Arresting such blackouts has remained elusive - mainly because relays (protection devices) operate on local data, and are prone to hidden faults that are impossible to detect until they manifest, resulting in misoperations that have sometime been precipitators or contributors to blackouts. Inspired by the Presidential policy directive on resilience -- meaning the ability to anticipate, prepare, withstand, and recover from disruptive events, this project proposes "WARP: Wide Area assisted Resilient Protection", a paradigm that adds a layer of finer (supervisory) intelligence to supplement conventional protection wisdom - which we call resilient protection. Exploiting high fidelity measurements and computation to calculate and analyze energy function components of power systems to identify disturbances, WARP would allow relays to be supervised - correct operations would be corroborated, and misoperations will be remedied by judiciously reversing the relay operation in a rational time-frame. The project also envisions predicting instability using advanced estimation techniques, thus being proactive. This will provide power grid the ability to auto-correct and bounce back from misoperations, curtailing the size, scale and progression of blackouts and improving the robustness and resilience of the electric grid -- our nation's most critical infrastructure. In WARP, disruptive events are deciphered by using synchrophasor data, energy functions, and dynamic state information via particle filtering. The information is fused to provide a global data set and intelligence signal that supervises relays, and also to predict system stability. Resilience is achieved when the supervisory signal rectifies the misoperation of relays, or endorses their action when valid. This endows relays with post-event-auto-correct abilities 
- a feature that never been explored/understood in the protection-stability nexus. Architectures to study the effect of latency and bad data are proposed. WARP introduces new notions: global detectability and distinguishability for power system events, stability prediction based on
the sensitivity of the energy function components and uses a novel factorization method: (CUR) preserving data interpretability to reduce data dimensionality. All the proposed
tools will be wrapped into a simulation framework to assess scalability and accuracy-runtime tradeoffs, and quantify the degree of resilience achieved. The effectiveness of the proposed scheme during extreme events will be measured by reenacting two well-documented blackout sequences. In addition, simulations on benchmarked systems will be performed to assess scalability and accuracy-runtime tradeoffs, and quantify the degree of resilience achieved.

The wide-area measurement systems technology using Phasor Measurement Units (PMUs) has been regarded as the key to guaranteeing stability, reliability, state estimation, and control of next-generation power systems. However, with the exponentially increasing number of PMUs, and the resulting explosion in data volume, the design and deployment of an efficient wide-area communication and computing infrastructure is evolving as one of the greatest challenges to the power system and IT communities. The goal of this NSF CPS project is to address this challenge, and construct a massively deployable cyber-physical architecture for wide-area control that is fast, resilient and cost-optimal (FRESCO). The FRESCO grid will consist of a suite of optimal control algorithms for damping oscillations in power flows and voltages, implemented on top of a cost-effective and cyber-secure distributed computing infrastructure connected by high-speed wide-area networks that are dynamically programmable and reconfigurable. The value of constructing FRESCO is twofold (1) If a US-wide communication network capable of transporting gigabit volumes of PMU data for wide-area control indeed needs to be implemented over the next five years then power system operators must have a clear sense of how various forms of delays, packet losses, and security threats affect the stability of these control loops. (2) Moreover, such wide-area communication must be made economically feasible and sustainable via joint decision-making processes between participating utility companies, and testing how controls can play a potential role in facilitating such economics. Currently, there is very limited insight into how the PMU data transport protocols may lead to a variety of such delay patterns, or dictate the economic investments. FRESCO will answer all of these questions, starting from small prototypical grid models to those with tens of thousands of buses. Our eventual goal will be to make FRESCO fully open-source for Transition to Practice (TTP). We will work with two local software companies in Raleigh, namely Green Energy Corporation and Real-Time Innovations, Inc. to develop a scalable, secure middleware using Data-Distribution Service (DDS) technology. Thus, within the scope of the project, we also expect to enrich the state-of-the-art cloud computing and networking technologies with new control and management functions. From a technical perspective, FRESCO will answer three main research questions. First, can wide-area controllers be co-designed in sync with communication delays to make the closed-loop system resilient and delay-aware, rather than just delay-tolerant This is particularly important, as PMU data, in most practical scenarios, will have to be transported over a shared resource, sharing bandwidth with other ongoing applications, giving rise to not only transport delays, but also significant delays due to queuing and routing. Advanced ideas of arbitrated network control designs will be used to address this problem. The second question we address is for cost. Given that there are several participants in this wide-area control, how much is each participant willing to pay in sharing the network cost with others for the sake of supporting a system-wide control objective compared to its current practice of opting for selfish feedback control only Ideas from cooperative game theory will be used to investigate this problem. The final question addresses security how can one develop a scientific methodology to assess risks, and mitigate security attacks in wide-area control? Statistical and structural analysis of attack defense modes using Bayesian and Markov models, game theory, and discrete-event simulation will be used to address this issue. Experimental demos will be carried out using the DETER-WAMS network, showcasing the importance of cyber-innovation for the sustainability of energy infrastructures. Research results will be broadcast through journal publications, and jointly organized graduate courses between NCSU, MIT and USC.