Despite many advances in vehicle automation, much remains to be done: the best autonomous vehicle today still lags behind human drivers, and connected vehicle (V2V) and infrastructure (V2I) standards are only just emerging. In order for such cyber-physical systems to fully realize their potential, they must be capable of exploiting one of the richest and most complex abilities of humans, which we take for granted: seeing and understanding the visual world. If automated vehicles had this ability, they could drive more intelligently, and share information about road and environment conditions, events, and anomalies to improve situational awareness and safety for other automated vehicles as well as human drivers. That is the goal of this project, to achieve a synergy between computer vision, machine learning and cyber-physical systems that leads to a safer, cheaper and smarter transportation sector, and which has potential applications to other sectors including agriculture, food quality control and environment monitoring. To achieve this goal, this project brings together expertise in computer vision, sensing, embedded computing, machine learning, big data analytics and sensor networks to develop an integrated edge-cloud architecture for (1) "anytime scene understanding" to unify diverse scene understanding methods in computer vision, and (2) "cooperative scene understanding" that leverages vehicle-to-vehicle and vehicle-to-infrastructure protocols to coordinate with multiple systems, while (3) emphasizing how security and privacy should be managed at scale without impacting overall quality-of-service. This architecture can be used for autonomous driving and driver-assist systems, and can be embedded within infrastructure (digital signs, traffic lights) to avoid traffic congestion, reduce risk of pile-ups and improve situational awareness. Validation and transition of the research to practice are through integration within City of Pittsburgh public works department vehicles, Carnegie Mellon University NAVLAB autonomous vehicles, and across the smart road infrastructure corridor under development in Pittsburgh. The project also includes activities to foster development of a new cyber-physical systems workforce, though involvement of students in the research, co-taught multi-disciplinary courses, and co-organized workshops.

Advances in technology mean that computer-controlled physical devices that currently still require human operators, such as automobiles, trains, airplanes, and medical treatment systems, could operate entirely autonomously and make rational decisions on their own. Autonomous cars and drones are a concrete and highly publicized face of this dream. Before this dream can be realized we must address the need for safety - the guaranteed absence of undesirable behaviors emerging from autonomy. Highly publicized technology accidents such as rocket launch failures, uncontrolled exposure to radiation during treatment, aircraft automation failures and unintended automotive accelerations serve as warnings of what can happen if safety is not adequately addressed in the design of such cyber-physical systems. One approach for safety analysis is the use of software tools that apply formal logic to prove the absence of undesired behavior in the control software of a system. In prior work, this approach this been proven to work for simple controller software that is generated automatically by tools from abstract models like Simulink diagrams. However, autonomous decision making requires more complex software that is able to solve optimization problems in real time. Formal verification of control software that includes such optimization algorithms remains an unmet challenge. The project SORTIES (Semantics of Optimization for Real Time Intelligent Embedded Systems) draws upon expertise in optimization theory, control theory, and computer science to address this challenge. Beginning with the convergence properties of convex optimization algorithms, SORTIES examines how these properties can be automatically expressed as inductive invariants for the software implementation of the algorithms, and then incorporates these properties inside the source code itself as formal annotations which convey the underlying reasoning to the software engineer and to existing computer-aided verification tools. The SORTIES goal is an open-source-semantics-carrying autocoder, which takes an optimization algorithm and its convergence properties as input, and produces annotated, verifiable code as output. The demonstration of the tool on several examples, such as a Mars lander, an aircraft avionics system, and a jet engine controller, shows that the evidence of quality produced by annotations is fully compatible with its application to truly functional products. Project research is integrated with education through training of "tri-lingual" professionals, who are equally conversant in system operation, program analysis, and the theory of control and optimization.

In the next few decades, autonomous vehicles will become an integral part of the traffic flow on highways. However, they will constitute only a small fraction of all vehicles on the road. This research develops technologies to employ autonomous vehicles already in the stream to improve traffic flow of human-controlled vehicles. The goal is to mitigate undesirable jamming, traffic waves, and to ultimately reduce the fuel consumption. Contemporary control of traffic flow, such as ramp metering and variable speed limits, is largely limited to local and highly aggregate approaches. This research represents a step towards global control of traffic using a few autonomous vehicles, and it provides the mathematical, computational, and engineering structure to address and employ these new connections. Even if autonomous vehicles can provide only a small percentage reduction in fuel consumption, this will have a tremendous economic and environmental impact due to the heavy dependence of the transportation system on non-renewable fuels. The project is highly collaborative and interdisciplinary, involving personnel from different disciplines in engineering and mathematics. It includes the training of PhD students and a postdoctoral researcher, and outreach activities to disseminate traffic research to the broader public. This project develops new models, computational methods, software tools, and engineering solutions to employ autonomous vehicles to detect and mitigate traffic events that adversely affect fuel consumption and congestion. The approach is to combine the data measured by autonomous vehicles in the traffic flow, as well as other traffic data, with appropriate macroscopic traffic models to detect and predict congestion trends and events. Based on this information, the loop is closed by carefully following prescribed velocity controllers that are demonstrated to reduce congestion. These controllers require detection and response times that are beyond the limit of a human's ability. The choice of the best control strategy is determined via optimization approaches applied to the multiscale traffic model and suitable fuel consumption estimation. The communication between the autonomous vehicles, combined with the computational and control tasks on each individual vehicle, require a cyber-physical approach to the problem. This research considers new types of traffic models (micro-macro models, network approaches for higher-order models), new control algorithms for traffic flow regulation, and new sensing and control paradigms that are enabled by a small number of controllable systems available in a flow.

Large battery systems with 100s/1000s cells are being used to power various physical platforms. For example, automobiles are transitioning from conventional powertrains to (plug-in) hybrid and electric vehicles (EVs). To achieve the desired efficiency of EVs, significant improvements are needed in the architecture and algorithms of battery management. This project will develop a new comprehensive battery management architecture, called Smart Battery Management System (SBMS). The research is expected to bridge the wide gap existing between cyber-physical system (CPS) research and electrification industry communities, provide environment-friendly solutions, increase the awareness of CPS, and develop skilled human resources. This project will incorporate and enhance a battery management system (BMS) by including battery state-of-charge (SoC) and state-of-health (SoH) algorithms as well as power management strategies on both pack and cell levels. Specifically, it consists of five main research tasks: (i) design a dynamically reconfigurable energy storage system to tolerate harsh internal and external stresses; (ii) develop cell-level thermal management algorithms; (iii) develop efficient, dependable charge and discharge scheduling algorithms in hybrid energy storage systems; (iv) develop a comprehensive, diagnostic/prognostic (P/D) algorithm with system parameters adjusted for making optimal decisions; and (v) build a testbed and evaluate the proposed architecture and algorithms on the testbed. This research will advance the state-of-the-art in the management of large-scale energy storage systems, extending their life and operation-time significantly, which is key to a wide range of battery-powered physical platforms. That is, SBMS will enable batteries to withstand excessive stresses and power physical platforms for a much longer time, all at low costs. SBMS will also serve as a basic framework for various aspects of CPS research, integrating (cyber) dynamic control and P/D mechanisms, and (physical) energy storage system dynamics.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.

This cross-disciplinary project brings together a team of engineering and computer science researchers to create, validate, and demonstrate the value of new techniques for ensuring that systems composed of combinations of hardware, software, and humans are designed to operate in a truly synergistic and safe fashion. One notable and increasingly common feature of these "Cyber-Physical-Human" (CPH) systems is that the responsibility for safe operation and performance is typically shared by increasingly sophisticated automation in the form of hardware and software, and humans who direct and oversee the behavior of automation yet may need to intervene to take over manual or shared system control when unexpected environmental situations or hardware or software failures occur. The ultimate goal is to achieve levels of safety and performance in system operation that exceed the levels attainable by either skilled human operators or completely autonomous systems acting alone. To do so, the research team will draw upon their expertise in the design of robust, fault-tolerant control systems, in the design of complexity-reduction architectures for software verification, and in human factors techniques for cognitive modeling to assure high levels of human situation awareness through effective interface design. By doing so, the safety, cost and performance benefits of increasingly sophisticated automation can be achieved without the frequently observed safety risks caused by automation creating greater distance between human operators and system operation. The techniques will be iteratively created and empirically evaluated using experimentation in human-in-the-loop simulations, including a medium-fidelity aircraft and flight simulator and a simulation of assistive automation in a medical context. More broadly, this research is expected to impact and inform the engineering of future CPH systems generally, for all industries and systems characterized by an increasing use of hardware and software automation directed and overseen by humans who provide an additional layer of safety in expected situations, Examples include highway and automotive automation, aerospace and air traffic control automation, semi-automated process control systems, and the many forms of automated systems and devices increasingly being used in medical contexts, such as the ICU and operating room. This research is also expected to inform government and industry efforts to provide safety certification criteria for the technologies used in CPH systems, and to educate a next generation of students trained in the cross-disciplinary skills and abilities needed to engineer the CPH systems of the future. The investigators will organize industry, academic, and government workshops to disseminate results and mentor students who are members of underrepresented groups through the course of this research project.

Cyber-physical systems employed in transportation, security and manufacturing applications rely on a wide variety of sensors for prediction and control. In many of these systems, acquisition of information requires the deployment and activation of physical sensors, which can result in increased expense or delay. A fundamental aspect of these systems is that they must seek information intelligently in order to support their mission, and must determine the optimal tradeoffs as to the cost of physical measurements versus the improvement in information. A recent explosion in sensor and UAV technology has led to new capabilities for controlling the nature and mobility of sensing actions by changing excitation levels, position, orientation, sensitivity, and similar parameters. This has in turn created substantial challenges to develop cyber-physical systems that can effectively exploit the degrees of freedom in selecting where and how to sense the environment. These challenges include high-dimensionality of observations and the associated "curse of dimensionality", non-trivial relationships between the observations and the latent variables, poor understanding of models relating the nature of potential sensing actions and the corresponding value of the collected information, and lack of sufficient training data from which to learn these models. Intellectual Merit: The proposed research includes: (1) data-driven stochastic control theory for intelligent sensing in cyber-physical systems that incorporates costs/delays/risks and accounts for scenarios where models for sensing, decision-making, and prediction are unavailable or poorly understood. (2) Validation of control methods on a UAV sensor network in the real world domain of archaeological surveying. Broader Impacts: The proposed effort includes: (a) Outreach: planned efforts for encouraging participation of women and under-represented groups; (b) Societal impact: research will lead to novel concepts in environmental monitoring, traffic surveillance, and security applications. (c) Multi- disciplinary activities: Impacting existing knowledge in cyber-physical systems, sensor management, and statistical learning. Research findings will be disseminated through conferences presentations, departmental seminars, journal papers, workshops and special sessions at IEEE CDC and RSS; (d) Curriculum development through new graduate level courses and course projects.

To ensure operational safety of complex cyber-physical systems such as automobiles, aircraft, and medical devices, new models, analyses, platforms, and development techniques are needed that can predict, possible interactions between features, detect them in the features' concrete implementations, and either eliminate or mitigate such interactions through precise modeling and enforcement of mixed-criticality cyber-physical system semantics. This project is taking a novel approach to reasoning about and managing feature interactions in cyber-physical systems, which encompasses interactions within software, interactions through the physical dynamics of the system, and interactions via shared computational resources. The proposed approach consists of three tightly coupled research thrusts: (1) a novel way of modeling features as automata equipped with both physical dynamics of the feature environment, and an assigned criticality level in each state of an automaton, (2) new automata-theoretic and control-theoretic analysis techniques, enabled by the modeling approach, and (3) new algorithms for adaptive sharing of computational resources between individual features that are guaranteed to satisfy the assumptions made during analysis, realized within a novel mixed-criticality cyber-physical platform architecture. The modeling approach will introduce a new model for mixed-criticality cyber-physical components and will support modern development standards, such as AUTOSAR in the automotive industry, for assigning criticality levels to features. Component interfaces in this model will capture control modes and the associated physical dynamics, operating modes and the associated resource requirements and criticality level, as well as relationships between control modes and operating modes. Analysis of features expressed in the proposed model will include detection of interactions and exploration of their effect on safety properties of the composite system. The broader impacts of the proposed work are twofold. One impact lies in the pervasive use of cyber-physical systems in our society. If the developed results are adopted in industry, it may help to promote improved safety of such systems. Results of the proposed research will be used in courses offered at both University of Pennsylvania and Washington University at the graduate and undergraduate levels. The project will also provide students with opportunities to get involved in cutting edge research within their fields of study.

Until now, the "cyber" component of automobiles has consisted of control algorithms and associated software for vehicular subsystems designed to achieve one or more performance, efficiency, reliability, comfort, or safety goals, primarily based on short-term intrinsic vehicle sensor data. However, there exist many extrinsic factors that can affect the degree to which these goals can be achieved. These factors can be determined from: longer-term traces of in-built sensor data that can be abstracted as triplines, socialized versions of these that are shared amongst vehicle users, and online databases. These three sources of information collectively constitute the automotive infoverse. This project harnesses this automotive infoverse to achieve these goals through high-confidence vehicle tuning and driver feedback decisions. Specifically, the project develops software called Headlight that permits the rapid development of apps that use the infoverse to achieve one or more goals. Advisory apps can provide feedback to the driver in order to ensure better fuel efficiency, while auto-tuning goals can set car parameters to promote safety. Allowing vehicles and such apps to share vehicle data with others and to use extrinsic information results in novel information processing, assurance, and privacy challenges. The project develops methods, algorithms and models to address these challenges. Broader Impact - This project can have significant societal impact by reducing carbon emissions and improving vehicular safety, can spur innovation in tuning methods and encourage researchers to experiment with this class of cyber-physical systems. The active participation of General Motors will strongly facilitate technology transfer. There is significant outreach including high school student participation, undergraduate research activities, internships, and creation of an open framework for plug and play application developers to use.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control. The proposed work will develop a new theoretical framework for stochastic learning, decision-making, and control in stochastically-varying cyber physical systems. In order to obtain analytical insights into the structure of efficient design, we first consider the case where the actions of the cyber system only affect the estimate of the underlying physical system. This class of problems arises in the context of (distributed) sensing/tracking of a physical system in isolation from cyber system control of the physical system's state. Joint state estimation and control for cyber-physical systems will then be considered. Here the most natural first step is to obtain sufficient conditions and/or special classes of systems where a separated approach to the information acquisition and efficient control is (near) optimal. To demonstrate its utility in practice, our theoretical framework will be applied in the specific context of energy efficient control of data centers and robust control of the smart grid under limited sensing. The intellectual merit of this work will be to develop a theoretical framework for the design of cyber-physical systems including information acquisition, state estimation, and control. In addition, separation theorems for the optimality of separate state estimation and control will be explored. In terms of broader impacts, significant performance improvement of control systems closed over communication networks will impact a wide range of applications for societal benefit, including smart buildings, intelligent transportation systems, energy-efficient data centers, and the future smart-grid. The PIs plan to disseminate the research results widely through conferences and journals, as well as by organizing specialized workshops and conference sessions related to cyber physical systems. The proposed project will train Ph.D. students as well as enrich the curriculum taught by the PIs in communications, stochastic control, and networks. The PIs have a strong track record in diversity and outreach activities, which for this project will include exposure and involvement of high school and undergraduate students, including under-represented minorities and women.