CPS: Synergy: Collaborative Research: Formal Models of Human Control and Interaction with Cyber-Physical Systems
Lead PI:
Meeko Oishi
Abstract
Cyber-Physical Systems (CPS) encompass a large variety of systems including for example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. One of the most important challenges in the design and deployment of Cyber-Physical Systems is how to formally guarantee that they are amenable to effective human control. This is a challenging problem not only because of the operational changes and increasing complexity of future CPS but also because of the nonlinear nature of the human-CPS system under realistic assumptions. Current state of the art has in general produced simplified models and has not fully considered realistic assumptions about system and environmental constraints or human cognitive abilities and limitations. To overcome current state of the art limitations, our overall research goal is to develop a theoretical framework for complex human-CPS that enables formal analysis and verification to ensure stability of the overall system operation as well as avoidance of unsafe operating states. To analyze a human-CPS involving a human operator(s) with bounded rationality three key questions are identified: (a) Are the inputs available to the operator sufficient to generate desirable behaviors for the CPS? (b) If so, how easy is it for the operator with her cognitive limitations to drive the system towards a desired behavior? (c) How can areas of poor system performance and determine appropriate mitigations be formally identified? The overall technical approach will be to (a) develop and appropriately leverage general cognitive models that incorporate human limitations and capabilities, (b) develop methods to abstract cognitive models to yield tractable analytical human models (c) develop innovative techniques to design the abstract interface between the human and underlying system to reflect mutual constraints, and (d) extend current state-of-the-art reachability and verification algorithms for analysis of abstract interfaces, iin which one of the systems in the feedback loop (i.e., the user) is mostly unknown, uncertain, highly variable or poorly modeled. The research will provide contributions with broad significance in the following areas: (1) fundamental principles and algorithms that would serve as a foundation for provably safe robust hybrid control systems for mixed human-CPS (2) methods for the development of analytical human models that incorporate cognitive abilities and limitations and their consequences in human control of CPS, (3) validated techniques for interface design that enables effective human situation awareness through an interface that ensures minimum information necessary for the human to safely control the CPS, (4) new reachability analysis techniques that are scalable and allow rapid determination of different levels of system safety. The research will help to identify problems (such as automation surprises, inadequate or excessive information contained in the user interface) in safety critical, high-risk, or expensive CPS before they are built, tested and deployed. The research will provide the formal foundations for understanding and developing human-CPS and will have a broad range of applications in the domains of healthcare, energy, air traffic control, transportation systems, homeland security and large-scale emergency response. The research will contribute to the advancement of under-represented students in STEM fields through educational innovation and outreach. The code, benchmarks and data will be released via the project website. Formal descriptions of models of human cognition are in general incompatible with formal models of the Cyber Physical System (CPS) the human operator(s) control. Therefore, it is difficult to determine in a rigorous way whether a CPS controlled by a human operator will be safe or stable and under which circumstances. The objective of this research is to develop an analytic framework of human-CPS systems that encompasses engineering compatible formal models of the human operator that preserve the basic architectural features of human cognition. In this project the team will develop methodologies for building such models as well as techniques for formal verification of the human-CPS system so that performance guarantees can be provided. They will validate models in a variety of domains ranging from air traffic control to large scale emergency response to the administration of anesthesia.
Meeko Oishi

Meeko Oishi received the Ph.D. (2004) and M.S. (2000) in Mechanical Engineering from Stanford University (Ph.D. minor, Electrical Engineering), and a B.S.E. in Mechanical Engineering from Princeton University (1998). She is a Professor of Electrical and Computer Engineering at the University of New Mexico. Her research interests include human-centric control, stochastic optimal control, and autonomous systems. She previously held a faculty position at the University of British Columbia at Vancouver, and postdoctoral positions at Sandia National Laboratories and at the National Ecological Observatory Network. She was a Visiting Researcher at AFRL Space Vehicles Directorate, and a Science and Technology Policy Fellow at The National Academies. She is the recipient of the NSF CAREER Award and a member of the 2021-2023 DoD Defense Science Study Group.

Performance Period: 09/15/2013 - 08/31/2016
Institution: University of New Mexico
Sponsor: National Science Foundation
Award Number: 1329878
Project URL
CPS: Synergy: Plug-and-Play Cyber-Physical Systems to Enable Intelligent Buildings
Lead PI:
Jianghai Hu
Abstract
1329875 (Hu). Despite their importance within the energy sector, buildings have not kept pace with technological improvements and particularly the introduction of intelligent features. A primary obstacle in enabling intelligent buildings is their highly distributed and diffuse nature. To address this challenge, a modular approach will be investigated for building design, construction, and operation that would completely transform the building industry. Buildings would be assembled from a set of pre-engineered intelligent modules and commissioned on site in a "plug-and-play" manner much like a "LEGO" set but with added capability of (a) allowing for easy configuration and re-configuration that can be integrated to provide delivery of thermal and visual comfort, ventilation; (b) providing optimized controls in terms of overall occupant satisfaction and energy efficiency and performance monitoring. The primary goal of the research is to develop and demonstrate innovative concepts for distributed intelligence along with a new paradigm for plug-and-play building control that is a necessary precursor in enabling this transformation. To accomplish these tasks, the investigators constitute a multidisciplinary team with expertise from three engineering disciplines, namely Civil (Architectural), Mechanical, Electrical and Computer Engineering. The intellectual merit of this research lies in developing a unified approach that advances the engineering of cyber-physical systems (CPS) for buildings by contributing to the following fields: (a) modeling and identification of building subsystems and integrated systems; (b) multi-agent system networks that enable distributed intelligent monitoring and control of multi-zone buildings; (c) optimal control algorithms for stochastic hybrid systems that can optimize the operation of buildings with mode changes under uncertainty. These contributions will be integrated in simulation and experimental platforms for multi-agent building system networks to validate the developed algorithms and to provide a new CPS-based technological solution to the control and optimization of modular buildings. An initial knowledge/technology base will be provided for scalable, adaptive, robust, and efficient engineering solutions for cyber-enabled building systems that will transform the current building operation practice, enabling the next generation of smart buildings with optimized comfort delivery and energy use. The broader impacts of this project are: (a) Theoretical development of modeling representations, algorithms, and simulation tools that will impact a number of scientific communities, including Civil/Architectural, Mechanical and Computer Engineering, Computer Science, and Operations Research. The proposed new principles for heterogeneous multi-agent system networks, distributed intelligence, and optimal hybrid control algorithms will have impacts in a diverse range of fields outside of building systems such as power systems, transportation systems, robotics, etc.; (b) Integration of the proposed modeling, simulation, and experimental platforms into new teaching modules and experiential learning activities that support the curriculum development in three engineering schools and Purdue?s first year engineering program; (c) Dissemination of research outcomes to the industry to open up a new horizon of business and economy that would enable the growth of green and intelligent buildings; (d) The creation of outreach and engagement initiatives that motivate K-12 teachers and students in STEM learning and research, broaden the participation of underrepresented groups in engineering, and motivate undergraduate students to participate in research related to emerging CPS topics.
Performance Period: 01/01/2014 - 12/31/2016
Institution: Purdue University
Sponsor: National Science Foundation
Award Number: 1329875
CPS: Synergy: Collaborative Research: Engineering Safety-Critical Cyber-Physical-Human Systems
Lead PI:
Xiaofeng Wang
Abstract
This cross-disciplinary project brings together a team of engineering and computer science researchers to create, validate, and demonstrate the value of new techniques for ensuring that systems composed of combinations of hardware, software, and humans are designed to operate in a truly synergistic and safe fashion. One notable and increasingly common feature of these "Cyber-Physical-Human" (CPH) systems is that the responsibility for safe operation and performance is typically shared by increasingly sophisticated automation in the form of hardware and software, and humans who direct and oversee the behavior of automation yet may need to intervene to take over manual or shared system control when unexpected environmental situations or hardware or software failures occur. The ultimate goal is to achieve levels of safety and performance in system operation that exceed the levels attainable by either skilled human operators or completely autonomous systems acting alone. To do so, the research team will draw upon their expertise in the design of robust, fault-tolerant control systems, in the design of complexity-reduction architectures for software verification, and in human factors techniques for cognitive modeling to assure high levels of human situation awareness through effective interface design. By doing so, the safety, cost and performance benefits of increasingly sophisticated automation can be achieved without the frequently observed safety risks caused by automation creating greater distance between human operators and system operation. The techniques will be iteratively created and empirically evaluated using experimentation in human-in-the-loop simulations, including a medium-fidelity aircraft and flight simulator and a simulation of assistive automation in a medical context. More broadly, this research is expected to impact and inform the engineering of future CPH systems generally, for all industries and systems characterized by an increasing use of hardware and software automation directed and overseen by humans who provide an additional layer of safety in expected situations, Examples include highway and automotive automation, aerospace and air traffic control automation, semi-automated process control systems, and the many forms of automated systems and devices increasingly being used in medical contexts, such as the ICU and operating room. This research is also expected to inform government and industry efforts to provide safety certification criteria for the technologies used in CPH systems, and to educate a next generation of students trained in the cross-disciplinary skills and abilities needed to engineer the CPH systems of the future. The investigators will organize industry, academic, and government workshops to disseminate results and mentor students who are members of underrepresented groups through the course of this research project.
Performance Period: 10/01/2013 - 09/30/2016
Institution: University of South Carolina at Columbia
Sponsor: National Science Foundation
Award Number: 1329870
CPS: Synergy: Collaborative Research: Safety-Feature Modeling and Adaptive Resource Management for Mixed-Criticality Cyber-Physical Systems
Lead PI:
Christopher Gill
Co-PI:
Abstract
To ensure operational safety of complex cyber-physical systems such as automobiles, aircraft, and medical devices, new models, analyses, platforms, and development techniques are needed that can predict, possible interactions between features, detect them in the features' concrete implementations, and either eliminate or mitigate such interactions through precise modeling and enforcement of mixed-criticality cyber-physical system semantics. This project is taking a novel approach to reasoning about and managing feature interactions in cyber-physical systems, which encompasses interactions within software, interactions through the physical dynamics of the system, and interactions via shared computational resources. The proposed approach consists of three tightly coupled research thrusts: (1) a novel way of modeling features as automata equipped with both physical dynamics of the feature environment, and an assigned criticality level in each state of an automaton, (2) new automata-theoretic and control-theoretic analysis techniques, enabled by the modeling approach, and (3) new algorithms for adaptive sharing of computational resources between individual features that are guaranteed to satisfy the assumptions made during analysis, realized within a novel mixed-criticality cyber-physical platform architecture. The modeling approach will introduce a new model for mixed-criticality cyber-physical components and will support modern development standards, such as AUTOSAR in the automotive industry, for assigning criticality levels to features. Component interfaces in this model will capture control modes and the associated physical dynamics, operating modes and the associated resource requirements and criticality level, as well as relationships between control modes and operating modes. Analysis of features expressed in the proposed model will include detection of interactions and exploration of their effect on safety properties of the composite system. The broader impacts of the proposed work are twofold. One impact lies in the pervasive use of cyber-physical systems in our society. If the developed results are adopted in industry, it may help to promote improved safety of such systems. Results of the proposed research will be used in courses offered at both University of Pennsylvania and Washington University at the graduate and undergraduate levels. The project will also provide students with opportunities to get involved in cutting edge research within their fields of study
Performance Period: 10/01/2013 - 09/30/2017
Institution: Washington University in St. Louis
Sponsor: National Science Foundation
Award Number: 1329861
CPS: Synergy: Collaborative Research: Thermal-Aware Management of Cyber-Physical Systems
Lead PI:
C.Mani Krishna
Co-PI:
Abstract
Processors in cyber-physical systems are increasingly being used in applications where they must operate in harsh ambient conditions and a computational workload which can lead to high chip temperatures. Examples include cars, robots, aircraft and spacecraft. High operating temperatures accelerate the aging of the chips, thus increasing transient and permanent failure rates. Current ways to deal with this mostly turn off the processor core or drastically slow it down when some part of it is seen to exceed a given temperature threshold. However, this pass/fail approach ignores the fact that (a) processors experience accelerated aging due to high temperatures, even if these are below the threshold, and (b) while deadlines are a constraint for real-time tasks to keep the controlled plant in the allowed state space, the actual controller response times that will increase if the voltage or frequency is lowered (to cool down the chip) are what determine the controlled plant performance. Existing approaches also fail to exploit the tradeoff between controller reliability (affected by its temperature history) and the performance of the plant. This project addresses these issues. Load-shaping algorithms are being devised to manage thermal stresses while ensuring appropriate levels of control quality. Such actions include task migration, changing execution speed, selecting an alternative algorithm or software implementation of control functions, and terminating prematurely optional portions of iterative tasks. Validation platforms for this project include automobiles and unmanned aerial vehicles. These platforms have been chosen based on both their importance to society and the significant technical challenges they pose. With CPS becoming ever more important in our lives and businesses, this project which will make CPS controllers more reliable and/or economical has broad potential social and economic impacts. Collaboration with General Motors promotes transition of the new technology to industry. The project includes activities to introduce students to thermal control in computing, in courses spanning high-school, undergraduate and graduate curricula.
Performance Period: 10/01/2013 - 09/30/2016
Institution: University of Massachusetts Amherst
Sponsor: National Science Foundation
Award Number: 1329831
CPS: Synergy: A Novel Biomechatronic Interface Based on Wearable Dynamic Imaging Sensors
Lead PI:
Siddhartha Sikdar
Co-PI:
Abstract
The problem of controlling biomechatronic systems, such as multiarticulating prosthetic hands, involves unique challenges in the science and engineering of Cyber Physical Systems (CPS), requiring integration between computational systems for recognizing human functional activity and intent and controlling prosthetic devices to interact with the physical world. Research on this problem has been limited by the difficulties in noninvasively acquiring robust biosignals that allow intuitive and reliable control of multiple degrees of freedom (DoF). The objective of this research is to investigate a new sensing paradigm based on ultrasonic imaging of dynamic muscle activity. The synergistic research plan will integrate novel imaging technologies, new computational methods for activity recognition and learning, and high-performance embedded computing to enable robust and intuitive control of dexterous prosthetic hands with multiple DoF. The interdisciplinary research team involves collaboration between biomedical engineers, electrical engineers and computer scientists. The specific aims are to: (1) research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity (2) develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor and (3) perform experiments to evaluate the real-time control of a prosthetic hand. The proposed research methods are broadly applicable to assistive technologies where physical systems, computational frameworks and low-power embedded computing serve to augment human activities or to replace lost functionality. The research will advance CPS science and engineering through integration of portable sensors for image-based sensing of complex adaptive physical phenomena such as dynamic neuromuscular activity, and real-time sophisticated image understanding algorithms to interpret such phenomena running on low-power high performance embedded systems. The technological advances would enable practical wearable image-based biosensing, with applications in healthcare, and the computational methods would be broadly applicable to problems involving activity recognition from spatiotemporal image data, such as surveillance. This research will have societal impacts as well as train students in interdisciplinary methods relevant to CPS. About 1.6 million Americans live with amputations that significantly affect activities of daily living. The proposed project has the long-term potential to significantly improve functionality of upper extremity prostheses, improve quality of life of amputees, and increase the acceptance of prosthetic limbs. This research could also facilitate intelligent assistive devices for more targeted neurorehabilitation of stroke victims. This project will provide immersive interdisciplinary CPS-relevant training for graduate and undergraduate students to integrate computational methods with imaging, processor architectures, human functional activity and artificial devices for solving challenging public health problems. A strong emphasis will be placed on involving undergraduate students in research as part of structured programs at our institution. The research team will involve students with disabilities in research activities by leveraging an ongoing NSF-funded project. Bioengineering training activities will be part of a newly developed undergraduate curriculum and a graduate curriculum under development. The synergistic research plan has been designed to advance CPS science and engineering through the development of new computational methods for dynamic activity recognition and learning from image sequences, development of novel wearable imaging technologies including high-performance embedded computing, and real-time control of a physical system. The specific aims are to: (1) Research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity. The first aim has three subtasks designed to collect, analyze and understand image sequences associated with functional tasks. (2) Develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor. The second aim has two subtasks designed to integrate wearable imaging sensors with a real-time computational platform. (3) Perform experiments to evaluate the real-time control of a prosthetic hand. The third aim will integrate the wearable image acquisition system developed in Aim 2, and the image understanding algorithms developed in Aim 1, for real-time evaluation of the control of a prosthetic hand interacting with a virtual reality environment. Successful completion of these aims will result in a real-time system that acquires image data from complex neuromuscular activity, decodes activity intent from spatiotemporal image data using computational algorithms, and controls a prosthetic limb in a virtual reality environment in real time. Once developed and validated, this system can be the starting point for developing a new class of sophisticated control algorithms for intuitive control of advanced prosthetic limbs, new assistive technologies for neurorehabilitation, and wearable real-time imaging systems for smart health applications.
Performance Period: 02/01/2014 - 01/31/2018
Institution: George Mason University
Sponsor: National Science Foundation
Award Number: 1329829
CPS: Synergy: Collaborative Research: Event-Based Information Acquisition, Learning, and Control in High-Dimensional Cyber-Physical Systems
Lead PI:
Tara Javidi
Abstract
This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.
Performance Period: 10/01/2013 - 09/30/2016
Institution: University of California at San Diego
Sponsor: National Science Foundation
Award Number: 1329819
CPS: Breakthrough: Scalable Component-Based Model Revision of Cyber-Physical Systems with Separation of Concerns
Lead PI:
Sandeep Kulkarni
Abstract
This project develops algorithms for revising a given model for a cyber-physical system while ensuring that the revised model is correct-by-construction and is realizable in the constraints imposed by the cyber-physical system. It specializes these algorithms in the context of fault-tolerance (with the theory of separation of concerns) and in the context of timed models (with the role of fairness). The project identifies constraints imposed by the inability to revise some or all physical components and ensure that they are satisfied during revision. It specializes model revision algorithms in two contexts: fault-tolerance and role of fairness during revision. Regarding fault-tolerance, it develops the theory of separation of concerns for cyber-physical systems. This work bridges the gap between fault-tolerance components, control theory and model revision. Regarding fairness, it develops efficient algorithms for revision by using abstraction to model continuous behaviors with discrete behaviors that utilize fairness. One broad impact of this project is to advance the fundamental science and technology of cyber-physical systems by developing systematic methods that ensure system correctness during maintenance where the system is revised due to changing requirements and/or environment. The algorithms from this project will provide techniques for providing assurance in automotive and aeronautical systems. In the context where fault-tolerance properties are added, the proposed activities also have the potential to identify missing specifications early and thereby reduce the cost of designing corresponding systems. The proposed activities facilitate in educating graduate students about different tasks involved in providing assurance via component based models and via model revision.
Performance Period: 10/01/2013 - 09/30/2016
Institution: Michigan State University
Sponsor: National Science Foundation
Award Number: 1329807
CPS: Synergy: Collaborative Research: Diagnostics and Prognostics Using Temporal Causal Models for Cyber Physical Systems- A Case of Smart Electric Grid
Lead PI:
Gabor Karsai
Co-PI:
Abstract
Reliable operation of cyber-physical systems (CPS) of societal importance such as Smart Electric Grids is critical for the seamless functioning of a vibrant economy. Sustained power outages can lead to major disruptions over large areas costing millions of dollars. Efficient computational techniques and tools that curtail such systematic failures by performing fault diagnosis and prognostics are therefore necessary. The Smart Electric Grid is a CPS: it consists of networks of physical components (including generation, transmission, and distribution facilities) interfaced with cyber components (such as intelligent sensors, communication networks, and control software). This grant provides funding to develop new methods to build models for the smart grid representing the failure dependencies in the physical and cyber components. The models will be used to build an integrated system-wide solution for diagnosing faults and predicting future failure propagations that can account for existing protection mechanisms. The original contribution of this work will be in the integrated modeling of failures on multiple levels in a large distributed cyber-physical system and the development of novel, hierarchical, robust, online algorithms for diagnostics and prognostics. If successful, the model-based fault diagnostics and prognostics techniques will improve the effectiveness of isolating failures in large systems by identifying impending failure propagations and determining the time to critical failures that will increase system reliability and reduce the losses accrued due to failures. This work will bridge the gap between fault management approaches used in computer science and power engineering that are needed as the grid becomes smarter, more complex, and more data intensive. Outcomes of this project will include modeling and run-time software prototypes, research publications, and experimental results in collaborations with industry partners that will be made available to the scientific community.
Performance Period: 10/01/2013 - 09/30/2016
Institution: Vanderbilt University
Sponsor: National Science Foundation
Award Number: 1329803
CPS: Synergy: Collaborative Research: Diagnostics and Prognostics Using Temporal Causal Models for Cyber Physical Systems- A Case of Smart Electric Grid
Lead PI:
Array Array
Abstract
Reliable operation of cyber-physical systems (CPS) of societal importance such as Smart Electric Grids is critical for the seamless functioning of a vibrant economy. Sustained power outages can lead to major disruptions over large areas costing millions of dollars. Efficient computational techniques and tools that curtail such systematic failures by performing fault diagnosis and prognostics are therefore necessary. The Smart Electric Grid is a CPS: it consists of networks of physical components (including generation, transmission, and distribution facilities) interfaced with cyber components (such as intelligent sensors, communication networks, and control software). This grant provides funding to develop new methods to build models for the smart grid representing the failure dependencies in the physical and cyber components. The models will be used to build an integrated system-wide solution for diagnosing faults and predicting future failure propagations that can account for existing protection mechanisms. The original contribution of this work will be in the integrated modeling of failures on multiple levels in a large distributed cyber-physical system and the development of novel, hierarchical, robust, online algorithms for diagnostics and prognostics. If successful, the model-based fault diagnostics and prognostics techniques will improve the effectiveness of isolating failures in large systems by identifying impending failure propagations and determining the time to critical failures that will increase system reliability and reduce the losses accrued due to failures. This work will bridge the gap between fault management approaches used in computer science and power engineering that are needed as the grid becomes smarter, more complex, and more data intensive. Outcomes of this project will include modeling and run-time software prototypes, research publications, and experimental results in collaborations with industry partners that will be made available to the scientific community.
Performance Period: 10/01/2013 - 09/30/2016
Institution: North Carolina State University
Sponsor: National Science Foundation
Award Number: 1329800
Subscribe to