This project investigates new reinforcement learning algorithms to enable long-term real-time autonomous learning by cyber-physical systems (CPS). The complexity of CPS makes hand-programming safe and efficient controllers for them difficult. For CPS to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes with potential for solving this problem.

This project develops the theory and technology for a new frontier in cyber-physical systems: cyber-physical manipulation. The goal of cyber-physical manipulation is to enable groups of hundreds or thousands of individual robotic agents to collaboratively explore an environment, manipulate objects in that environment, and transport those objects to desired locations.

This project establishes a new framework for the formal verification of cyber-physical systems. The framework combines the power of logical decision engines and scalable numerical methods to perform safety verification of general nonlinear hybrid systems. The key difficulty with formal verification of hybrid systems is that all scalable modern verification techniques rely heavily on the use of powerful decision procedures. For hybrid systems, one needs to reason about logic formulas over the real numbers with nonlinear functions, which has been regarded as an intractable problem.

Cyber-physical systems employed in transportation, security and manufacturing applications rely on a wide variety of sensors for prediction and control. In many of these systems, acquisition of information requires the deployment and activation of physical sensors, which can result in increased expense or delay.

Cyber-physical systems are found in nearly every area of daily life: transportation, energy, medical systems, and food production. Life and safety frequently depend upon their correct operation. This project develops a novel systematic framework and methods for understanding, designing, and controlling complex coupled cyber and physical systems based on large-scale computation. This is achieved by explicitly developing the connection between the abstraction, modeling and verification frameworks of physics-based models and those of discrete-transition systems.

Cyber-Physical Systems (CPS) encompass a large variety of systems including for example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. One of the most important challenges in the design and deployment of Cyber-Physical Systems is how to formally guarantee that they are amenable to effective human control.

To ensure operational safety of complex cyber-physical systems such as automobiles, aircraft, and medical devices, new models, analyses, platforms, and development techniques are needed that can predict, possible interactions between features, detect them in the features' concrete implementations, and either eliminate or mitigate such interactions through precise modeling and enforcement of mixed-criticality cyber-physical system semantics.

This grant provides funding for establishing the scientific foundations of a product innovation process that can engage a vastly larger pool of talent to generate new ideas and to create new cyber-physical products. The primary objective is to address fundamental issues pertaining to natural interfaces, behavioral modeling and secure knowledge sharing, with particular emphasis on their integration.

Until now, the "cyber" component of automobiles has consisted of control algorithms and associated software for vehicular subsystems designed to achieve one or more performance, efficiency, reliability, comfort, or safety goals, primarily based on short-term intrinsic vehicle sensor data. However, there exist many extrinsic factors that can affect the degree to which these goals can be achieved.

This project focuses on the problem of information acquisition, state estimation and control in the context of cyber physical systems. In our underlying model, a (set of) decision maker(s), by controlling a sequence of actions with uncertain outcomes, dynamically refines the belief about stochastically time-varying parameters of interest. These parameters are then used to control the physical system efficiently and robustly. Here the cyber system collects, processes, and acquires information about the underlying physical system of interest, which is used for its control.