It is now recognized that cloud data centers are a significant consumer of energy resources and a substantial source of greenhouse gas emission. On the other hand, the intermittency and uncertainty of renewable energy present a daunting operating challenge for the electricity grid. The key idea behind this CRII: Cyber-Physical Systems project is that these two challenges are in fact symbiotic: data centers can be virtual batteries for the electricity grid. Specifically, data centers are large loads, but are also flexible. If the electricity grid can call on the flexibilities of data centers via appropriate demand response programs, this will be a crucial tool for easing the incorporation of renewable energy into the electricity grid. Unfortunately, despite the great potential, the current reality is that data centers perform little demand response. This project aims at the interdisciplinary challenges of enabling demand response from cloud data centers to realize the societal benefits. The overarching goal of this project is to develop an intellectual framework to understand and guide the realization of demand response from cloud data centers, to address engineering and economic challenges in order to manage the daunting risk. This project will first quantify the potential economic and environmental benefits of demand response from cloud data centers. The quantification includes the societal cost savings and emission reductions from networked data centers through geographical load balancing, and the impacts of demand response taxonomy. Built upon the first thrust, this project will continue to tackle the interdisciplinary challenges of both local control algorithm design and global market design for data center demand response in order to facilitate their participation in various demand response programs. The researchers will study prediction-based pricing design and analysis, demand response program design based on optimization decomposition, and distributed online algorithm design for risk management and distributed control. The results of this project will, at the societal level, help utility companies and load serving entities realize the great potential that lies in the Cloud, and, furthermore, design demand response programs that provide right incentives for data center operators to participate. At a local level, this project will help guide the management of geographically distributed data centers in participating in the right demand response programs. The control algorithms and demand response programs, as well as the methodology, can be applied beyond data centers. This research will create new knowledge in distributed online algorithm design and optimization-based market design. Additionally, this project will help design an interdisciplinary course Sustainable IT and IT for Sustainability. Personnel involved in this project, graduates and undergraduates, will receive innovation experiences through the algorithm design, analysis, implementation, and testing.

This proposal will establish a framework for developing distributed Cyber-Physical Systems operating in a Networked Control Systems (NCS) environment. Specific attention is focused on an application where the computational, and communication challenges are unique due to the sheer size of the physical system, and communications between system elements include potential for significant losses and delays. An example of this is the power grid which includes large-scale deployment of distributed and networked Phasor Measurement Units (PMUs) and wind energy resources. Although, much has been done to model and analyze the impact of data dropouts and delay in NCS at a theoretical level, their impact on the behavior of cyber physical systems has received little attention. As a result much of the past research done on the `smart grid' has oversimplified the `physical' portion of the model, thereby overlooking key computational challenges lying at the heart of the dimensionality of the model and the heterogeneity in the dynamics of the grid. A clear gap has remained in understanding the implications of uncertainties in NCS (e.g. bandwidth limitations, packet dropout, packet disorientation, latency, signal loss, etc.) cross-coupled with the uncertainties in a large power grid with wind farms (e.g. variability in wind power, fault and nonlinearity, change in topology etc.) on the reliable operation of the grid. To address these challenges, this project will, for the first time, develop a modeling framework for discovering hitherto unknown interactions through co-simulation of NCS, distributed computing, and a large power grid included distributed wind generation resources. Most importantly, it addresses challenges in distributed computation through frequency domain abstractions and proposes two novel techniques in grid stabilization during packet dropout. The broader impact lies in providing deeper understanding of the impact of delays and dropouts in the Smart Grid. This will enable a better utilization of energy transmission assets and improve integration of renewable energy sources. The project will facilitate participation of women in STEM disciplines, and will include outreach with local Native American tribal community colleges This project will develop fundamental understanding of impact of network delays and drops using an approach that is applicable to a variety of CPS. It will enable transformative Wide-Areas Measurement Systems research for the smart grid through modeling adequacy studies of a representative sub-transient model of the grid along with the representation of packet drop in the communication network by a Gilbert model. Most importantly, fundamental concepts of frequency domain abstraction including balanced truncation and optimal Hankel-norm approximation are proposed to significantly reduce the burden of distributed computing. Finally, a novel `reduced copy' approach and a `modified Kalman filtering' approach are proposed to address the problem of grid stabilization using wind farm controls when packet drop is encountered.

This CAREER project responds to an urgent need to develop mobile power distribution systems that lower deployment and operating costs while simultaneously increasing network efficiency and response in dynamic and often dangerous physical conditions. The significant need for an efficient and effective mobile power distribution system became evident during search and rescue/recovery missions following the Japan tsunami and the disappearance of the Malaysia MH370 airplane. The technology outcomes from this project will apply to a broad range of environments (in space, air, water or on ground) where the success of long-term robotic network missions is measured by the ability of the robots to operate, for an extended period of time, in highly dynamic and potentially hazardous environments. These advanced features will provide the following advantages: efficiency, efficacy, guaranteed persistence, enhanced performance, and increased success in search/rescue/recovery/discovery missions. Specifically, this project addresses the following technology problems as it translates from research discovery toward commercial application: inflated energy use currently required when the autonomous vehicles break from mission to return to recharging station; lack of multi-robot coordination needed to take into account both fundamental hardware and network science challenges necessary to respond to energy needs and dynamic environment conditions. By addressing these gaps in technology, this work establishes the theoretical, computational, and experimental foundation for mobile power delivery and onsite recharging capability. Moreover, the new technology developed in this project is universally adaptable for disparate autonomous vehicles especially autonomous underwater vehicles (AUVs). In more technical terms, this project creates network optimization and formation strategies that will enable a power distribution system to reconfigure itself depending on the number of operational autonomous vehicles and recharging specifications to meet overall mission specifications, the energy consumption needs of the network, situational conditions, and environmental variables. Such a system will play a vital role in real-time controlled applications across multiple disciplines such as sensor networks, robotics, and transportation systems where limited power resources and unknown environmental dynamics pose major constraints. In addition to addressing technology gaps, undergraduate and graduate students will be involved in this research and will receive interdisciplinary education/ innovation/ technology translation/ outreach experiences through: developing efficient network energy routing, path planning and coordination strategies; designing and creating experimental test-beds and educational platforms; and engaging K-12th grade students in Science, Technology, Engineering and Math including those from underrepresented groups. This project engages Michigan Tech's Great Lake Research Center (GLRC) and Center for Agile Interconnected Microgrids (AIM) to develop experimental test-beds and conduct tests that validate the resulting methods and algorithms, and ultimately, facilitate the technology translation effort from research discovery toward commercial reality.

This cross-disciplinary research proposes a patient-specific cost-saving approach to the design and optimization of healthcare cyber-physical systems (HCPS). The HCPS computes the patient's physiological state based on sensors, communicates this information via a network from home to hospital for quantifying risk indices, signals the need for critical medical intervention in real time, and controls vital health signals (e.g., cardiac rhythm, blood glucose). The research proposed under the HCPS paradigm will treat the human body as a complex system. It will entail the development of mathematical models that capture the time-dependence and fractal behavior of physiological processes and the design of quality-of-life (QoL) control strategies for medical devices. The research will advance the understanding of the correlations between physiological processes, drug treatment, stress level and lifestyle. To date, the complex interdependence, variability and individual characteristics of physiological processes have not been taken into account in the design of medical devices and artificial organs. The existing mathematical approaches rely on reductionist and Markovian assumptions. This research project will rethink the theoretical foundations for the design of healthcare cyber-physical systems by capturing the interdependencies and fractal characteristics of physiological processes within a highly dynamic network. To establish the theoretical foundations of HCPS, a three-step approach will be followed: (i) construct a multi-scale non-equilibrium statistical physics inspired framework for patient modeling that captures the time dependence, non-Gaussian behavior, interdependencies and multi-fractal behavior of physiological processes; (ii) develop adaptive patient-specific and physiology-aware (multi-fractal) close-loop control algorithms for dynamic complex networks; (iii) design algorithms and methodologies for the HCPS networked components that account for biological and technological constraints. This research will significantly contribute to early chronic disease detection and treatment. Models and implementable algorithms, which can both predict physiological dynamics and assess the risk of acute and chronic diseases, will be valuable instruments for patient-centered healthcare. This in-depth mathematical analysis of physiological complexity facilitates a transformative multimodal and multi-scale approach to CPS design with healthcare applications. The project not only addresses the current scientific and technological gap in CPS, but can also foster new research directions in related fields such as the study of interdependent networks with implications for understanding homeostasis and diseases and the study and control of complex systems. The cyber-physical systems designed under this newly proposed paradigm will have vital social and economic implications, including the improvement of QoL and the reduction of lost productivity rates due to chronic diseases. The project will offer interdisciplinary training for graduate, undergraduate and K-12 students. The PI will integrate the research results within his courses at University of Southern California and make them widely available through the project website. Moreover, the PI will enhance civic engagement by involving college and K-12 students in community outreach activities that will raise awareness of the important role of health monitoring.

Airborne networking, unlike the networking of fixed sensors, mobile devices, and slowly-moving vehicles, is very challenging because of the high mobility, stringent safety requirements, and uncertain airspace environment. Airborne networking is important because of the growing complexity of the National Airspace System with the integration of unmanned aerial vehicles (UAVs). This project develops an innovative new theoretical framework for cyber-physical systems (CPS) to enable airborne networking, which utilizes direct flight-to-to-flight communication for flexible information sharing, safe maneuvering, and coordination of time-critical missions. This project uses an innovative co-design approach that exploits the mutual benefits of networking and decentralized mobility control in an uncertain heterogeneous environment. The approach departs from the usual perspective that views physical mobility as communication constraints, communication as constraints for decentralized mobility control, and uncertain environment as constraints for both. Instead, approach taken here proactively exploits the constraints, uncertainty, and new structures with information to enable high-performance designs. The features of the co-design such as scalability, fast response, trackability, and robustness to uncertainty advance the core CPS science on decision-making for large-scale networks under uncertainty. The technological advances developed in this research will contribute to multiple fields, including mobile networking, decentralized control, experiment design, and general real-time decision making under uncertainty for CPS. Technology transfer will be pursued through close collaboration with industries and national laboratories. This novel research direction will also serve as a unique backdrop to inspire the CPS workforce. New teaching materials will benefit the future CPS workforce by equipping them with a knowledge base in networking and control. Broad outreach and dissemination activities that involve undergraduate student societies, K-12 school teaching, and public events, all stemming from the PI's current efforts, will be enhanced.

This project advances the scientific knowledge on design methods for improving the resilience of civil infrastructures to disruptions. To improve resilience, critical services in civil infrastructure sectors must utilize new diagnostic tools and control algorithms that ensure survivability in the presence of both security attacks and random faults, and also include the models of incentives of human decision makers in the design process. This project will develop a practical design toolkit and platform to enable the integration of resiliency-improving control tools and incentive schemes for Cyber-Physical Systems (CPS) deployed in civil infrastructures. Theory and algorithms will be applied to assess resiliency levels, select strategies to improve performance, and provide reliability and security guarantees for sector-specific CPS functionalities in water, electricity distribution and transportation infrastructures. The main focus is on resilient design of network control functionalities to address problems of incident response, demand management, and supply uncertainties. More broadly, the knowledge and tools from this project will influence CPS designs in water, transport, and energy sectors, and also be applicable to other systems such as supply-chains for food, oil and gas. The proposed platform will be used to develop case studies, test implementations, and design projects for supporting education and outreach activities. Current CPS deployments lack integrated components designed to survive in uncertain environments subject to random events and the actions of strategic entities. The toolkit (i) models the propagation of disruptions due to failure of cyber-physical components, (ii) detects and responds to both local and network-level failures, and (iii) designs incentive schemes that improve aggregate levels of public good (e.g., decongestion, security), while accounting for network interdependencies and private information among strategic entities. The validation approach uses real-world data collected from public sources, test cases developed by domain experts, and simulation software. These tools are integrated to provide a multi-layer design platform, which explores the design space to synthesize solutions that meet resiliency specifications. The platform ensures that synthesized implementations meet functionality requirements, and also estimates the performance guarantees necessary for CPS resilience. This modeling, validation, exploration, and synthesis approach provides a scientific basis for resilience engineering. It supports CPS education by providing a platform and structured workflow for future engineers to approach and appreciate implementation realities and socio-technical constraints.

Every year around 30,000 fatalities and 2.2 million injuries happen on US roads. The problem is compounded with huge economic losses due to traffic congestions. Advances in Cooperative Vehicle Efficiency and Safety (CVES) systems promise to significantly reduce the human and economic cost of transportation. However, large scale deployment of such systems is impeded by significant technical and scientific gaps, especially when it comes to achieving real-time and high accuracy situational awareness for cooperating vehicles. This CAREER project aims at closing these gaps through developing fundamental information networking methodologies for coordinated control of automated systems. These methodologies will be based on the innovative concept of modeled knowledge propagation. In addition, the educational component of this project integrates interdisciplinary Cyber-Physical Systems (CPS) subjects on the design of automated networked systems into graduate and undergraduate training modules. For robust operation, CVES systems require each vehicle to have reliable real-time awareness of the state of other coordinated vehicles. This project addresses the critical need for robust control-oriented situational awareness by developing a multi-resolution information networking methodology that is model- and context-aware. The approach is to develop the novel concepts of model communication and its derived multi-resolution networking. Context-aware model-communication relies on transmission and synchronization of models (e.g., stochastic hybrid system structures and parameters) instead of raw measurements. This allows for high fidelity synchronization of dynamical models of CVES over networks. Multi-resolution networking concept is enabled through scalable representations of models. Multi resolution models allow in-network adaptation of model fidelity to available network resources. The result is robustness of CVES to network service variability. The successful deployment of CVES, even partially, will provide significant societal benefits through reduced traffic accidents and improved efficiency. This project will enable large scale CVES deployment by addressing its scalability challenge. In addition, methodologies developed in this project will be crucial to emerging autonomous vehicles, which are also expected to coordinate their actions over communication networks. The fundamental research outcomes on knowledge propagation through network synchronization of dynamical models will be broadly applicable in other CPS domains such as smart grid. The educational component of this project will target training of CPS researchers and engineers on subjects in intelligent transportation and energy systems.

Cyber-physical critical infrastructures integrate networks of computational and physical processes to provide the society with essential services. The power grid, in particular, is a vast and interconnected cyber-physical network for delivering electricity from generation plants to end-point consumers. Protecting power grid critical infrastructures is a vital necessity because the failure of these systems would have a debilitating impact on economic security and public health and safety. However, several recent large-scale outages and the significant increase in the number of major attacks over the past four years confirm the insufficiency of the current protection solutions for these systems. Existing tedious manual tolerance procedures cannot protect those grids against sophisticated attacks. Additionally, use of purely-cyber security solutions for power grid resiliency is not sufficient because they ignore the cyber-physical interdependencies, power-side sensor measurements, and the possibility of countermeasures in power infrastructures. The objective of this research is to investigate fundamental problems in cyber-physical tolerance and develop an integrated set of mathematically rigorous and real-world deployable capabilities, resulting in a system that can model, analyze, predict, and tolerate complex security incidents in computing, physical, or communication assets in a near-real-time manner. The proposed research will provide system administrators and power grid operators with scalable and online integrated cyber-physical monitoring and incident response capabilities through keeping track of cyber-physical infrastructure's dynamic evolution caused by distributed security incidents, optimal proactive response and recovery countermeasures and adaptive preparation for potential future security incidents. The proposed research will facilitate trustworthy operation of next-generation complex and large-scale power grids. The research outcomes will be integrated into educational and knowledge transfer initiatives that involves implementation of curricular activities, innovative learning game development, university workshops, and hands-on K-12 summer camps and academic-year high-school courses, as well as Industry technology transfer efforts to develop a workforce with the capability to reason across multiple disciplines. Through holistic consideration of both cyber and physical factors under adversarial situations, this fundamental work will be applicable to other cyber-physical domains and can transform the way people approach the problem of cyber-physical security.

The electric power grid, a cyber-physical system (CPS), faces an alarmingly high risk of catastrophic damage from cyber-attacks. However, modeling cyber-attacks, evaluating consequences, and developing appropriate countermeasures require a detailed, realistic, and tractable model of electric power CPS operations. The primary barrier is the lack of access to models for the complex legacy proprietary systems upon which the electric power grid has relied for decades. This project aims to overcome these challenges with the development of an attack-verifying (verifiable) software framework that will capture the electric power system operations in adequate detail. Cyber threats will be verified using this framework through a combination of sound theoretical methods and an open-source commercial simulation engine accessible via a unique transition to practice (TTP) option. This research focuses on four fundamental and related thrusts: (i) identifying classes of cyber-attacks with quantifiable physical consequences and developing detection-based countermeasures; (ii) identifying communication attacks on distributed grid operations and developing information-sharing countermeasures; (iii) developing a verifiable software framework that models the spatio-temporal operations of the electric grid in tandem with thrusts (i) and (ii) to verify attack models, evaluate countermeasures, and develop new resiliency protocols; and (iv) a TTP option, in collaboration with industry-leading experts from IncSys and PowerData, to develop commercial grade open source power simulation software packages to integrate and test the attacks and countermeasures of Thrusts (i) through (iii) as well as develop workforce training curriculum for North American Electric Reliability Council (NERC) certification. This research also includes engagement with K-12 students via the Arizona Science Laboratory program.

All cyber-physical systems (CPS) depend on properly calibrated sensors to sense the surrounding environment. Unfortunately, the current state of the art is that calibration is often a manual and expensive operation; moreover, many types of sensors, especially economical ones, must be recalibrated often. This is typically costly, performed in a lab environment, requiring that sensors be removed from service. MetaSense will reduce the cost and management burden of calibrating sensors. The basic idea is that if two sensors are co-located, then they should report similar values; if they do not, the least-recently-calibrated sensor is suspect. Building on this idea, this project will provide an autonomous system and a set of algorithms that will automate the detection of calibration issues and preform recalibration of sensors in the field, removing the need to take sensors offline and send them to a laboratory for calibration. The outcome of this project will transform the way sensors are engineered and deployed, increasing the scale of sensor network deployment. This in turn will increase the availability of environmental data for research, medical, personal, and business use. MetaSense researchers will leverage this new data to provide early warning for factors that could negatively affect health. In addition, graduate student engagement in the research will help to maintain the STEM pipeline. This project will leverage large networks of mobile sensors connected to the cloud. The cloud will enable using large data repositories and computational power to cross-reference data from different sensors and detect loss of calibration. The theory of calibration will go beyond classical models for computation and physics of CPS. The project will combine big data, machine learning, and analysis of the physics of sensors to calculate two factors that will be used in the calibration. First, MetaSense researchers will identify measurement transformations that, applied in software after the data collection, will generate calibrated results. Second, the researchers will compute the input for an on-board signal-conditioning circuit that will enable improving the sensitivity of the physical measurement. The project will contribute research results in multiple disciplines. In the field of software engineering, the project will contribute a new theory of service reconfiguration that will support new architecture and workflow languages. New technologies are needed because the recalibration will happen when the machine learning algorithms discover calibration errors, after the data has already been collected and processed. These technologies will support modifying not only the raw data in the database by applying new calibration corrections, but also the results of calculations that used the data. In the field of machine learning, the project will provide new algorithms for dealing with spatiotemporal maps of noisy sensor readings. In particular, the algorithms will work with Gaussian processes and the results of the research will provide more meaningful confidence intervals for these processes, substantially increasing the effectiveness of MetaSense models compared to the current state of the art. In the field of pervasive computing, the project will build on the existing techniques for context-aware sensing to increase the amount of information available to the machine learning algorithms for inferring calibration parameters. Adding information about the sensing context is paramount to achieve correct calibration results. For example, a sensor that measures air pollution inside a car on a highway will get very different readings if the car window is open or closed. Finally, the project will contribute innovations in sensor calibration hardware. Here, the project will contribute innovative signal-conditioning circuits that will interact with the cloud system and receive remote calibration parameters identified by the machine learning algorithms. This will be a substantial advance over current circuits based on simple feedback loops because it will have to account for the cloud and machine learning algorithms in the loop and will have to perform this more complex calibration with power and bandwidth constraints. Inclusion of graduate students in the research helps to maintain the STEM pipeline.