This project develops the theory and technology for a new frontier in cyber-physical systems: cyber-physical manipulation. The goal of cyber-physical manipulation is to enable groups of hundreds or thousands of individual robotic agents to collaboratively explore an environment, manipulate objects in that environment, and transport those objects to desired locations. The project embraces realistic assumptions about the communication, computation, and sensing capabilities of simple individual robots, leading to algorithmic solutions that intrinsically leverage population size in favor of complex agents. Cyber-physical solutions for locating, grasping, and characterizing objects require tools based on distributed computational geometry, while the tasks of planning a path, initiating motion, and controlling the trajectory require tools from decentralized control and consensus. The project lays the theoretical and algorithmic foundations of cyber-physical manipulation, and proves the feasibility of the concept experimentally in hardware tests with up to 100 individual robots. The project uses the problem of manipulation as a stage on which to explore the deeper cyber-physical issue of information asymmetry; the difference in the state of the world as perceived by different agents in the system due to differences in their history of observations, and limitations in their communication capabilities. The object retrieval problem studied in this project is an elemental building block for enabling more complex cyber-physical manipulation tasks. It provides crucial algorithmic components for numerous applications of broad societal benefit, including automated construction (in which hundreds or thousands of robots fabricate large, complex structures), autonomous emergency response (in which large teams of robots find and retrieve incapacitated human survivors after a disaster), and automated environmental cleanup (in which robots secure a dangerous environment by removing debris or hazardous substances). Furthermore, distributed algorithms for multi-agent systems are of broad scientific relevance beyond the realm of cyberphysical systems. The natural world is, in its algorithmic essence, decentralized at many levels. Hence, any advancement in the understanding of how groups of individual agents collaborate to accomplish a coherent task will have broad scientific ramifications. The project has a robust educational and outreach program. One aspect is a hands-on curriculum for robotics outreach activities, called the 'Cyber-Physical Manipulation Lab.' Using a custom-designed robot platform, this educational module introduces the theory and practice of cyber-physical systems to young students to attract them to STEM subject areas at an early age. Results of the project are also incorporated into several graduate and undergraduate level courses at Rice University and Boston University.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.

This cross-disciplinary project brings together a team of engineering and computer science researchers to create, validate, and demonstrate the value of new techniques for ensuring that systems composed of combinations of hardware, software, and humans are designed to operate in a truly synergistic and safe fashion. One notable and increasingly common feature of these "Cyber-Physical-Human" (CPH) systems is that the responsibility for safe operation and performance is typically shared by increasingly sophisticated automation in the form of hardware and software, and humans who direct and oversee the behavior of automation yet may need to intervene to take over manual or shared system control when unexpected environmental situations or hardware or software failures occur. The ultimate goal is to achieve levels of safety and performance in system operation that exceed the levels attainable by either skilled human operators or completely autonomous systems acting alone. To do so, the research team will draw upon their expertise in the design of robust, fault-tolerant control systems, in the design of complexity-reduction architectures for software verification, and in human factors techniques for cognitive modeling to assure high levels of human situation awareness through effective interface design. By doing so, the safety, cost and performance benefits of increasingly sophisticated automation can be achieved without the frequently observed safety risks caused by automation creating greater distance between human operators and system operation. The techniques will be iteratively created and empirically evaluated using experimentation in human-in-the-loop simulations, including a medium-fidelity aircraft and flight simulator and a simulation of assistive automation in a medical context. More broadly, this research is expected to impact and inform the engineering of future CPH systems generally, for all industries and systems characterized by an increasing use of hardware and software automation directed and overseen by humans who provide an additional layer of safety in expected situations, Examples include highway and automotive automation, aerospace and air traffic control automation, semi-automated process control systems, and the many forms of automated systems and devices increasingly being used in medical contexts, such as the ICU and operating room. This research is also expected to inform government and industry efforts to provide safety certification criteria for the technologies used in CPH systems, and to educate a next generation of students trained in the cross-disciplinary skills and abilities needed to engineer the CPH systems of the future. The investigators will organize industry, academic, and government workshops to disseminate results and mentor students who are members of underrepresented groups through the course of this research project.

This project investigates new reinforcement learning algorithms to enable long-term real-time autonomous learning by cyber-physical systems (CPS). The complexity of CPS makes hand-programming safe and efficient controllers for them difficult. For CPS to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes with potential for solving this problem. However, existing RL algorithms do not meet all of the requirements of learning in CPS. Efficacy of the new algorithms for CPS is evaluated in the context of smart buildings and autonomous vehicles. Cyber-physical systems (CPS) have the potential to revolutionize society by enabling smart buildings, transportation, medical technology, and electric grids. Success of this project could lead to a new generation of CPS that are capable of adapting to their situation and improving their performance autonomously over time. In addition to the traditional methods of dissemination, this project will develop and release open-source code implementing the new reinforcement learning algorithms. Education and outreach activities associated with the project include a Freshman Research Initiative course, participation in a UT Austin annual open house that draws in many underrepresented minorities to interest the public in computer science and science in general, and the department's annual summer school for high school girls called First Bytes.

This project develops the theory and technology for a new frontier in cyber-physical systems: cyber-physical manipulation. The goal of cyber-physical manipulation is to enable groups of hundreds or thousands of individual robotic agents to collaboratively explore an environment, manipulate objects in that environment, and transport those objects to desired locations. The project embraces realistic assumptions about the communication, computation, and sensing capabilities of simple individual robots, leading to algorithmic solutions that intrinsically leverage population size in favor of complex agents. Cyber-physical solutions for locating, grasping, and characterizing objects require tools based on distributed computational geometry, while the tasks of planning a path, initiating motion, and controlling the trajectory require tools from decentralized control and consensus. The project lays the theoretical and algorithmic foundations of cyber-physical manipulation, and proves the feasibility of the concept experimentally in hardware tests with up to 100 individual robots. The project uses the problem of manipulation as a stage on which to explore the deeper cyber-physical issue of information asymmetry; the difference in the state of the world as perceived by different agents in the system due to differences in their history of observations, and limitations in their communication capabilities. The object retrieval problem studied in this project is an elemental building block for enabling more complex cyber-physical manipulation tasks. It provides crucial algorithmic components for numerous applications of broad societal benefit, including automated construction (in which hundreds or thousands of robots fabricate large, complex structures), autonomous emergency response (in which large teams of robots find and retrieve incapacitated human survivors after a disaster), and automated environmental cleanup (in which robots secure a dangerous environment by removing debris or hazardous substances). Furthermore, distributed algorithms for multi-agent systems are of broad scientific relevance beyond the realm of cyberphysical systems. The natural world is, in its algorithmic essence, decentralized at many levels. Hence, any advancement in the understanding of how groups of individual agents collaborate to accomplish a coherent task will have broad scientific ramifications. The project has a robust educational and outreach program. One aspect is a hands-on curriculum for robotics outreach activities, called the 'Cyber-Physical Manipulation Lab.' Using a custom-designed robot platform, this educational module introduces the theory and practice of cyber-physical systems to young students to attract them to STEM subject areas at an early age. Results of the project are also incorporated into several graduate and undergraduate level courses at Rice University and Boston University.

Cyber-physical systems employed in transportation, security and manufacturing applications rely on a wide variety of sensors for prediction and control. In many of these systems, acquisition of information requires the deployment and activation of physical sensors, which can result in increased expense or delay. A fundamental aspect of these systems is that they must seek information intelligently in order to support their mission, and must determine the optimal tradeoffs as to the cost of physical measurements versus the improvement in information. A recent explosion in sensor and UAV technology has led to new capabilities for controlling the nature and mobility of sensing actions by changing excitation levels, position, orientation, sensitivity, and similar parameters. This has in turn created substantial challenges to develop cyber-physical systems that can effectively exploit the degrees of freedom in selecting where and how to sense the environment. These challenges include high-dimensionality of observations and the associated "curse of dimensionality", non-trivial relationships between the observations and the latent variables, poor understanding of models relating the nature of potential sensing actions and the corresponding value of the collected information, and lack of sufficient training data from which to learn these models. Intellectual Merit: The proposed research includes: (1) data-driven stochastic control theory for intelligent sensing in cyber-physical systems that incorporates costs/delays/risks and accounts for scenarios where models for sensing, decision-making, and prediction are unavailable or poorly understood. (2) Validation of control methods on a UAV sensor network in the real world domain of archaeological surveying. Broader Impacts: The proposed effort includes: (a) Outreach: planned efforts for encouraging participation of women and under-represented groups; (b) Societal impact: research will lead to novel concepts in environmental monitoring, traffic surveillance, and security applications. (c) Multi- disciplinary activities: Impacting existing knowledge in cyber-physical systems, sensor management, and statistical learning. Research findings will be disseminated through conferences presentations, departmental seminars, journal papers, workshops and special sessions at IEEE CDC and RSS; (d) Curriculum development through new graduate level courses and course projects.

Cyber-Physical Systems (CPS) encompass a large variety of systems including for example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. One of the most important challenges in the design and deployment of Cyber-Physical Systems is how to formally guarantee that they are amenable to effective human control. This is a challenging problem not only because of the operational changes and increasing complexity of future CPS but also because of the nonlinear nature of the human-CPS system under realistic assumptions. Current state of the art has in general produced simplified models and has not fully considered realistic assumptions about system and environmental constraints or human cognitive abilities and limitations. To overcome current state of the art limitations, our overall research goal is to develop a theoretical framework for complex human-CPS that enables formal analysis and verification to ensure stability of the overall system operation as well as avoidance of unsafe operating states. To analyze a human-CPS involving a human operator(s) with bounded rationality three key questions are identified: (a) Are the inputs available to the operator sufficient to generate desirable behaviors for the CPS? (b) If so, how easy is it for the operator with her cognitive limitations to drive the system towards a desired behavior? (c) How can areas of poor system performance and determine appropriate mitigations be formally identified? The overall technical approach will be to (a) develop and appropriately leverage general cognitive models that incorporate human limitations and capabilities, (b) develop methods to abstract cognitive models to yield tractable analytical human models (c) develop innovative techniques to design the abstract interface between the human and underlying system to reflect mutual constraints, and (d) extend current state-of-the-art reachability and verification algorithms for analysis of abstract interfaces, iin which one of the systems in the feedback loop (i.e., the user) is mostly unknown, uncertain, highly variable or poorly modeled. The research will provide contributions with broad significance in the following areas: (1) fundamental principles and algorithms that would serve as a foundation for provably safe robust hybrid control systems for mixed human-CPS (2) methods for the development of analytical human models that incorporate cognitive abilities and limitations and their consequences in human control of CPS, (3) validated techniques for interface design that enables effective human situation awareness through an interface that ensures minimum information necessary for the human to safely control the CPS, (4) new reachability analysis techniques that are scalable and allow rapid determination of different levels of system safety. The research will help to identify problems (such as automation surprises, inadequate or excessive information contained in the user interface) in safety critical, high-risk, or expensive CPS before they are built, tested and deployed. The research will provide the formal foundations for understanding and developing human-CPS and will have a broad range of applications in the domains of healthcare, energy, air traffic control, transportation systems, homeland security and large-scale emergency response. The research will contribute to the advancement of under-represented students in STEM fields through educational innovation and outreach. The code, benchmarks and data will be released via the project website. Formal descriptions of models of human cognition are in general incompatible with formal models of the Cyber Physical System (CPS) the human operator(s) control. Therefore, it is difficult to determine in a rigorous way whether a CPS controlled by a human operator will be safe or stable and under which circumstances. The objective of this research is to develop an analytic framework of human-CPS systems that encompasses engineering compatible formal models of the human operator that preserve the basic architectural features of human cognition. In this project the team will develop methodologies for building such models as well as techniques for formal verification of the human-CPS system so that performance guarantees can be provided. They will validate models in a variety of domains ranging from air traffic control to large scale emergency response to the administration of anesthesia.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.

The electric power grid is a complex cyber-physical system, whose reliable and secure operation is of paramount importance to national security and economic vitality. There is a growing and evolving threat of cyber-based attacks, both in numbers and sophistication, on the nation's critical infrastructure. Therefore, cyber security "encompassing attack prevention, detection, mitigation, and resilience" is critical in today's power grid and the emerging smart grid. The goal of this project is to develop a unified system-theoretic framework and analytical tools for cyber-physical security of power systems, capturing the dynamics of the physical system as well as that of the cyber system. Research tasks include: 1) Development of a methodology for impact analysis that includes systematic identification of worst-case stealthy attacks on the power system's wide-area control and evaluating the resulting consequences in terms of stability violations and performance loss. 2) Development of robust cyber-physical countermeasures, employing a combination of methods from system theory, cyber security, and model-based/data-driven tools, in the form of domain-specific anomaly detection/tolerance algorithms and attack-resilient control algorithms. 3) Evaluating the effectiveness of the proposed impact modeling and mitigation algorithms through a combination of simulation and testbed-based evaluations, using realistic system topologies and attack scenarios. The project makes significant contributions to enhance the security and resiliency of the power grid and lays a scientific foundation for cyber-physical security of critical infrastructure. Also, the project develops novel curriculum modules, mentors graduate and undergraduate students including under-represented minorities, leverages industrial collaborations, and exposes high school students to cyber security concepts.

Objective: How much a person should be allowed to interact with a controlled machine? If that machine is safety critical, and if the computer that oversees its operation is essential to its operation and safety, the answer may be that the person should not be allowed to interfere with its operation at all or very little. Moreover, whether the person is a novice or an expert matters. Intellectual Merit: This research algorithmically resolves the tension between the need for safety and the need for performance, something a person may be much more adept at improving than a machine. Using a combination of techniques from numerical methods, systems theory, machine learning, human-machine interfaces, optimal control, and formal verification, this research will develop a computable notion of trust that allows the embedded system to assess the safety of the instruction a person is providing. The interface for interacting with a machine matters as well; designing motions for safety-critical systems using a keyboard may be unintuitive and lead to unsafe commands because of its limitations, while other interfaces may be more intuitive but threaten the stability of a system because the person does not understand the needs of the system. Hence, the person needs to develop trust with the machine over a period of time, and the last part of the research will include evaluating a person's performance by verifying the safety of the instructions the person provides. As the person becomes better at safe operation, she will be given more authority to control the machine while never putting the system in danger. Broader Impacts: The activities will include outreach, development of public-domain software, experimental coursework including two massive online courses, and technology transfer to rehabilitation. Outreach will include exhibits at the Museum of Science and Industry and working with an inner-city high school. The algorithms to be developed will have immediate impact on projects with the Rehabilitation Institute of Chicago, including assistive devices, stroke assessment, and neuromuscular hand control. Providing a foundation for a science of trust has the potential to transform rehabilitation research.