This project aims to enable mutualistic interaction of cyber damage prognostics and physical reconfigurable sensing for mutualistic and self-adaptive cyber-physical systems (CPS). Drawing inspiration from mutualism in biology where two species interact in a way that benefits both, the cyber and the physical interact in a way that they simultaneously benefit from and contribute to each other to enhance the ability of the CPS to predict, reconfigure, and adapt. Such interaction is generalizable, allowing it to enhance CPS applications in various domains. In the civil infrastructure systems domain, the mutualistic interaction-enabled CPS will allow for reconfiguring a single type of sensor, adaptively based on damage prognostics, to monitor multiple classes of infrastructure damages ? thereby improving the cost-effectiveness of multi-damage infrastructure monitoring by reducing the types and number of sensors needed and maximizing the timeliness and accuracy of damage assessment and prediction at the same time. Enabling cost-effective multi-damage monitoring is promising to leapfrog the development of safer, more resilient, and sustainable infrastructure, which would stimulate economic growth and social welfare for the benefit of the nation and its people. This project will also contribute to NSF?s commitment to broadening participation in engineering (BPE) by developing innovative, interdisciplinary, and inclusive BPE programs to attract, train, and reward the next-generation engineering researchers and practitioners who are capable creators of CPS technology and not only passive consumers, thereby enhancing the U.S. economy, security, and well-being.
The envisioned CPS includes three integrated components: (1) data-driven, knowledge-informed deep learning methods for generalizable damage prognostics to predict the onset and propagation of infrastructure damages, providing information about target damages to inform reconfigurable sensing, (2) signal difference maximization theory-based reconfigurable sensing methods to optimize and physically control the configurations of the sensors to actively seek to monitor each of the predicted target damages, providing damage-seeking feedback to inform damage prognostics, and (3) quality-aware edge cloud computing methods for efficient and effective damage information extraction from raw sensing signals, serving as the bridge between damage prognostics and reconfigurable sensing. The proposed CPS will be tested in multi-damage monitoring of bridges using simulation-based and actual CPS prototypes, and would be generalized to monitoring other civil infrastructure in the future. The proposed CPS methods have the potential to transform the way we design, create, and operate CPS to enable the next-generation CPS that have greater predictive ability, reconfigurability, and adaptability.
Many cyber-physical systems (CPS) have real-time (RT) requirements. For these RT-CPS, such as a network of unmanned aerial vehicles that deliver packages to customers? homes or a robot that performs/aides in cardiac surgery, deadline misses may result in economic losses or even fatal consequences. At the same time, as these RT-CPS interact with, and are depended on by, humans, they must also be trustworthy. The goal of this research is to design secure RT-CPS that are less complex, easier to analyze, and reliable for critical application domains such as defense, medicine, transportation, manufacturing, and agriculture, to name just a few. Since RT-CPS now permeate most aspects of our daily lives, especially in the smart city and internet-of-things (IoT) context, this research will improve confidence in automated systems by users. Research results will be disseminated to both academia and industry, and permit timely adoption since the hardware required in this research is already publicly available. This project will result in a pipeline of engineers and computer scientists who are well-versed in the interdisciplinary nature of securing RT-CPS, as well as course modules and red-teaming exercises for undergraduate students in all engineering disciplines and interactive learning modules and internship experience for K-12 students in D.C., Detroit, Dallas, and St. Louis.
The goal of this research is to design secure RT-CPS from the ground up while explicitly accounting for physical dynamics of said RT-CPS at runtime to achieve resilience via prevention and detection of, and recovery from, attacks. This will be accomplished by (i) securing the scheduling infrastructure from the ground up, (ii) using a formal framework for trading off security against timeliness while accounting for system dynamics, and for the cost of security to be explicitly quantified, and (iii) performing state- and function-dependent on-demand recovery. Said RT-CPS will be able to proactively prevent attacks using moving target defenses, as well as detect and recover from attacks that cannot be avoided. This research will pave the way for RT-CPS and internet-of-things (IoT) to be implemented with confidence: their timely and correct operation guaranteed. Specific contributions of this research are: (i) a trusted scheduling infrastructure that can protect the integrity of the real-time tasks, the scheduler, its task queues, and I/O, and which can recover from (intentional) errors, (ii) a probabilistic real-time/security co-design framework that exploits trusted execution to protect the security of the real-time tasks, (iii) novel schedulability analysis techniques, (iv) an incremental recovery mechanism for continuous operation, and (v) validation on automated ground vehicles, drones, and robot arms. Contributions expanding the knowledge base will be made to the fields of CPS, IoT, real-time systems, security, and control systems.
Artificial Intelligence (AI) has shown superior performance in enhancing driving safety in advanced driver-assistance systems (ADAS). State-of-the-art deep neural networks (DNNs) achieve high accuracy at the expense of increased model complexity, which raises the computation burden of onboard processing units of vehicles for ADAS inference tasks. The primary goal of this project is to develop innovative collaborative AI inference strategies with the emerging edge computing paradigm. The strategies can adaptively adjust cooperative inference techniques for best utilizing available computation and communication resources and ultimately enable high-accuracy and real-time inference. The project will inspire greater collaborations between experts in wireless communication, edge computing, computer vision, autonomous driving testbed development, and automotive manufacturing, and facilitate AI applications in a variety of IoT systems. The educational testbed developed from this project can be integrated into courses to provide hands-on experiences. This project will benefit undergraduate, master, and Ph.D. programs and increase under-represented groups? engagement by leveraging the existing diversity-related outreach efforts.
A multi-disciplinary team with complementary expertise from Rowan University, Temple University, Stony Brook University, and Kettering University is assembled to pursue a coordinated study of collaborative AI inference. The PIs explore integrative research to enable deep learning technologies in resource-constrained ADAS for high-accuracy and real-time inference. Theory-wise, the PIs plan to take advantage of the observation that DNNs can be decomposed into a set of fine-grained components to allow distributed AI inference on both the vehicle and edge server sides for inference acceleration. Application-wise, the PIs plan to design novel DNN models which are optimized for the cooperative AI inference paradigm. Testbed-wise, a vehicle edge computing platform with V2X communication and edge computing capability will be developed at Kettering University GM Mobility Research Center. The cooperative AI inference system will be implemented, and the research findings will be validated on realistic vehicular edge computing environments thoroughly. The data, software, and educational testbeds developed from this project will be widely disseminated. Domain experts in autonomous driving testbed development, intelligent transportation systems, and automotive manufacturing will be engaged in project-related issues to ensure relevant challenges in this project are impactful for real-world applications.
The proposed research focuses on cascading failures in electrical energy cyber-physical systems (CPS), which is a critical infrastructure of our nation. Cascading failures, where the failure of one or few components causes a wide-spread failure of the interconnected system, is a major cause of blackouts in power grids. The mechanism of such failures is highly complex as it involves the physical layer of the grid (e.g. generators, transmission lines, etc.) and the cyber layer (e.g. communication and control elements) in a coupled manner. This is a very important problem to investigate as cascading failures can cost our economy billions of dollars. This project takes a holistic view at taming cascading failures in electrical energy CPS. The proposed research has two tightly coupled thrust areas. Thrust 1 aims at an accurate understanding of the cascading failure mechanism and its prevention, while Thrust 2 focuses on recovery following blackouts under uncertainty of failure locations. Theory of trajectory sensitivity and graph theory are leveraged to develop a fundamental understanding of cascading failures in energy CPS, which can be applied to other CPSs where the physical system is dynamic in nature and the failure propagation in the physical system and the cyber system are coupled. The proposed preventive control strategy can protect critical infrastructures from large-scale failures and facilitate higher resiliency, whereas the proposed recovery strategy is applicable in the aftermath of a blackout caused by cascades, natural disasters, or other events, which will reduce downtime of the critical infrastructure. In support of the Broadening Participation in Computing initiative among women, the proposed research will be integrated into the one-week summer camps offered by the School of EECS at Penn State. Presentations about this research will be given to high school girls over the course of one week in the 2019 camps, and then camps focused on curriculum on the topic of this research will be offered in 2020 and 2021.
The proposed research has two key objectives (a) develop an accurate understanding of the cascading failure mechanism and its prevention, and (b) develop a recovery plan following blackouts under uncertainty of failure locations and budget constraints. The quasi-steady-state (QSS) model of power grid used in literature for studying cascade propagation produces inaccurate results towards the later stages of blackouts, whereas a fully dynamic model is impractical for large-scale statistical analyses. To solve this, a 'temporally hybrid' and a 'spatio-temporally hybrid' model are proposed, which quantify the stress of the grid at the systems level and the component level, respectively, using trajectory sensitivity theory, and appropriately switch from the QSS to the dynamic model. Next, a unified graph-based model for interdependent power grid and communication systems is developed, which takes into account several special features of the legacy Supervisory Control and Data Acquisition (SCADA) system along with the modern Wide-Area Monitoring, Protection, and Controls (WAMPAC) system, and the observability and controllability they provide for the CPS. Furthermore, a stability-constrained remedial action scheme for cascade prevention is proposed. Finally, a new approach for progressive assessment and recovery, which leverages the hybrid power grid models and the unified communication network model, is proposed in the presence of budget constraints and failure uncertainties.
Consider two future autonomous system use-cases: (i) a bomb defusing rover sent into an unfamiliar, GPS and communication denied environment (e.g., a cave or mine), tasked with the objective of locating and defusing an improvised explosive device, and (ii) an autonomous racing drone competing in a future autonomous incarnation of the Drone Racing League. Both systems will make decisions based on inputs from a combination of simple, single output sensing devices, such as inertial measurement units, and complex, high dimensional output sensing modalities, such as cameras and LiDAR. This shift from relying only on simple, single output sensing devices to systems that incorporate rich, complex perceptual sensing modalities requires rethinking the design of safety-critical autonomous systems, especially given the inextricable role that machine and deep learning play in the design of modern perceptual sensors. These two motivating examples raise an even more fundamental question however: given the vastly different dynamics, environments, objectives, and safety/risk constraints, should these two systems have perceptual sensors with different properties? Indeed, due to the extremely safety critical nature of the bomb defusing task, an emphasis on robustness, risk aversion, and safety seems necessary. Conversely, the designer of the drone racer may be willing to sacrifice robustness to maximize responsiveness and lower lap-time. This extreme diversity in requirements highlights the need for a principled approach to navigate tradeoffs in this complex design space, which is what this proposal seeks to develop. Existing approaches to designing perception/action pipelines are either modular, which often ignore uncertainty and limit interaction between components, or monolithic and end-to-end, which are difficult to interpret, troubleshoot, and have high sample-complexity.
This project proposes an alternative approach and rethinks the scientific foundations of using machine learning and computer vision to process rich high-dimensional perceptual data for use in safety-critical cyber-physical control applications. Thrusts will develop integration between perception, planning and control that allow for their co-design and co-optimization. Using novel robust learning methods for perceptual representations and predictive models that characterize tradeoffs between robustness (e.g., to lighting & weather changes, rotations) and performance (e.g., responsiveness, discriminativeness), jointly learned perception maps and uncertainty profiles will be abstracted as ``noisy virtual sensors? for use in uncertainty aware perception-based planning & control algorithms with stability, performance, and safety guarantees. These insights will be integrated into novel perception-based model predictive control algorithms, which allow for planning, stability, and safety guarantees through a unifying optimization-based framework acting on rich perceptual data. Experimental validation of the benefits of these methods will be conducted at Penn using photorealistic simulations and physical camera equipped quadcopters, and be used to demonstrate perception-based planning and control algorithms at the extremes of speed/safety tradeoffs. On the educational front, the research outcomes of this proposal will be used to develop a sequence of courses on safe autonomy, safe perception, and learning and control at the University of Pennsylvania. Longer term, the goal of this project is to create a new community of researchers that focus on robust learning for perception-based control. Towards this goal, departmental efforts will be leveraged to increase and diversify the PhD students working on this project.
The future of cyber-physical systems are smart technologies that can work collaboratively, cooperatively, and safely with humans. Smart technologies and humans will share autonomy, i.e., the right, obligation and ability to share control in order to meet their mutual objectives in the environment of operations. For example, surgical robots must interact with surgeons to increase their capabilities in performing high-precision surgeries, drones need to deliver packages to humans and places, and autonomous cars need to share roads with human-driven cars. In all such interactions, these systems must act safely despite the risks and uncertainties that are intrinsic with humans, technologies, and the environments in which they interact. The key insight of this project is that control strategies can be developed that increase safety in situations where a human needs to closely interact with a cyber-physical system (CPS) that is capable of autonomy or semi-autonomous action.
The goal of this project is to develop risk-aware interactive control and planning for achieving safe cyber-physical-human (CPS-h) systems. This project will advance the state-of-the-art of CPS-h planning and control in three main ways: (i) developing computationally tractable risk-aware trajectory planning algorithms that are suited to general autonomous CPS-h, (ii) developing a computationally efficient and empirically supported framework to account for risk-awareness in human?s decision-making, and (iii) deriving interaction-aware planning algorithms for achieving safe and efficient interactions between multiple risk-aware agents. The proposed algorithms will be extensively evaluated with human subjects in interaction with autonomous CPS-h such as autonomous cars and quadcopters. This work will have direct impact on many CPS-h domains including but not limited to multi-agent interactions, autonomous driving, collaboration and coordination between humans and autonomous agents in safety-critical scenarios.
This project will develop the theory and algorithmic tools for the design of provably-safe controllers that can leverage preview information from different sources. Many autonomous or semi-autonomous cyber-physical systems (CPS) are equipped with mechanisms that provide a window of projecting into the future. These mechanisms can be forward looking sensors like cameras (and corresponding perception algorithms), map information, forecast information, or more complicated predictive models of external agents learned from data. Through these mechanisms, at run-time, the systems have a preview of what lies ahead. Leveraging this information to improve performance of CPS while keeping strong guarantees on their safety, therefore, holds great promise for multiple technologies of national interest. We will use driver-assist systems in connected vehicles as the main application. Education and outreach activities will involve undergraduate and graduate students along with stakeholders from local automotive companies.
To develop the theory for learning- and prediction-enabled safety for CPS we will: (i) develop a modeling formalism, namely preview automata, for systems with preview information and correct-by-construction control algorithms that consider structured inaccuracies in the predictions for resilience; (ii) investigate how cooperation can assist in enriching the preview information; (iii) learn, via finite-sample data analysis, trustworthy dynamical models of the behaviors of non-cooperative agents with provable uncertainty bounds; and (iv) design methods for selecting compatible models from the learned dynamical models and for deriving safe controllers in the presence of cooperative and non-cooperative agents. Our innovations will enable safety-critical CPS to take full advantage of emerging technologies on sensing, perception, communication, and learning.
Many cyber-physical systems (CPS) have real-time (RT) requirements. For these RT-CPS, such as a network of unmanned aerial vehicles that deliver packages to customers? homes or a robot that performs/aides in cardiac surgery, deadline misses may result in economic losses or even fatal consequences. At the same time, as these RT-CPS interact with, and are depended on by, humans, they must also be trustworthy. The goal of this research is to design secure RT-CPS that are less complex, easier to analyze, and reliable for critical application domains such as defense, medicine, transportation, manufacturing, and agriculture, to name just a few. Since RT-CPS now permeate most aspects of our daily lives, especially in the smart city and internet-of-things (IoT) context, this research will improve confidence in automated systems by users. Research results will be disseminated to both academia and industry, and permit timely adoption since the hardware required in this research is already publicly available. This project will result in a pipeline of engineers and computer scientists who are well-versed in the interdisciplinary nature of securing RT-CPS, as well as course modules and red-teaming exercises for undergraduate students in all engineering disciplines and interactive learning modules and internship experience for K-12 students in D.C., Detroit, Dallas, and St. Louis.
The goal of this research is to design secure RT-CPS from the ground up while explicitly accounting for physical dynamics of said RT-CPS at runtime to achieve resilience via prevention and detection of, and recovery from, attacks. This will be accomplished by (i) securing the scheduling infrastructure from the ground up, (ii) using a formal framework for trading off security against timeliness while accounting for system dynamics, and for the cost of security to be explicitly quantified, and (iii) performing state- and function-dependent on-demand recovery. Said RT-CPS will be able to proactively prevent attacks using moving target defenses, as well as detect and recover from attacks that cannot be avoided. This research will pave the way for RT-CPS and internet-of-things (IoT) to be implemented with confidence: their timely and correct operation guaranteed. Specific contributions of this research are: (i) a trusted scheduling infrastructure that can protect the integrity of the real-time tasks, the scheduler, its task queues, and I/O, and which can recover from (intentional) errors, (ii) a probabilistic real-time/security co-design framework that exploits trusted execution to protect the security of the real-time tasks, (iii) novel schedulability analysis techniques, (iv) an incremental recovery mechanism for continuous operation, and (v) validation on automated ground vehicles, drones, and robot arms. Contributions expanding the knowledge base will be made to the fields of CPS, IoT, real-time systems, security, and control systems.
Infants and toddlers with developmental disabilities or delays use early intervention (EI) for rehabilitation services. Yet, EI quality is compromised for racially and ethnically diverse and socially disadvantaged families. A key lever to improve EI quality is family-centered care, an evidence-based approach that is grounded in family engagement for shared decision-making. This project is motivated by the need to give families a smart and connected option for engaging in the design of the EI service plan for their child. This effort will develop and evaluate an upgraded Participation and Environment Measure (PEM), an evidence-based electronic option for directing equitable family-centered EI service design. PEM upgrades will (a) increase content relevance for racially and ethnically diverse families, and (b) leverage modern artificial intelligence solutions to personalize the PEM user experience to a broader range of EI enrolled families. This upgraded PEM electronic option will be evaluated in a population of racially and ethnically diverse EI families, to assess for its capacity to improve EI quality and to appraise supports and barriers to its longer-term implementation within the broader EI service system. This project builds evidence for the first customized, culturally relevant electronic option to direct family-centered care during EI service design. The approaches and technologies developed may be applicable to similar service contexts. Additionally, this project increases opportunities for conducting interdisciplinary research at the intersection of computer science and rehabilitation science, building interprofessional capacity for research engagement among EI service providers and students training for pediatric rehabilitation careers, and sponsoring students from historically underrepresented groups in diverse research labs that value inclusive excellence.
This proposal develops key innovations to family-centered EI in two ways. First, for the PEM electronic option, the project will (a) increase content relevance for racially and ethnically diverse families, and (b) personalize the PEM user experience to a broader range of EI enrolled families. For the former, the project will establish cultural equivalencies of the original PEM assessment and critically examine its intervention content to ensure that families can voice concerns about racial climate and collect and share goal attainment strategies using community-preferred communication channels. For the latter, the team will incorporate an adaptive conversational agent into the PEM intervention to improve caregiver navigation and guidance, and we will develop methods to automatically customize its strategy exchange feature to individual caregiver needs. These innovations will result in fundamental advances to natural language processing research through the investigation of adaptive dialogue policies for task-oriented or mixed-initiative dialogue systems, generalized dialogue act schema, and lexicon-informed meaning representations. We will evaluate the upgraded PEM electronic option with racially and ethnically diverse and socially disadvantaged EI enrolled families, to assess for its capacity to improve caregiver and provider perceptions of family-centered EI service quality, improve parent engagement in EI service plan implementation, and increase the availability and relevance of participation-focused EI service plans. We will engage EI stakeholders to appraise supports and barriers to its longer-term implementation in EI. These advances will yield evidence for a customized, culturally-relevant electronic option to foster family-centered care in EI.
Disasters disproportionately impact older adults who experience increased fatality rates; such individuals often live in age-friendly communities and senior health facilities (SHFs). During a crisis, older adults are often unable to shelter safely in place or self-evacuate due to a range of physical conditions (need for life-sustaining equipment, impaired mobility) and cognitive afflictions (e.g. dementia, Alzheimer?s). First responders assisting older adults could benefit from seamless, real-time access to critical life-saving information about the living facilities (e.g., floor plans, operational status, number of residents) and about individual residents (e.g., health conditions such as need for dialysis, oxygen, personal objects to reduce anxiety). Such information, siloed within organizational logs or held by caregivers, is inaccessible and/or unavailable at the time of response. This interdisciplinary project brings together IT, geriatrics and resilience experts with disaster-response agencies and SHF providers to create information preparedness and transform disaster resilience for older adults.
The team will design, develop and deploy CareDEX, a novel community contributed data-exchange platform, that empowers SHFs to readily assimilate, ingest, store and exchange information, both apriori and in real-time, with response agencies to care for older adults in extreme events. The CareDEX information pipeline enables SHFs to capture individual information about changing health conditions and personalized needs and share them with responders to help improve response. Information co-produced with civic partners will identify and refine resident-specific data via tools for proactive collection/update. Given the sensitive nature of personal information, e.g., health-profiles, CareDEX will incorporate policy-based information sharing mechanisms that balance needs for individual privacy with authorized information release. CareDEX?s hybrid-cloud architecture seamlessly enables data to be securely stored on-premise (at SHF) and in the cloud for remote access by responders and temporary caregivers. Relocation of older adults requires regional information (e.g. road-conditions, facility status) - CareDEX will integrate GIS tools to provide first-responders with uptodate region-level situational awareness for dynamic decision-support. The prototype CareDEX platform will be co-developed with core civic partners, e.g. Front Porch (a nation-wide senior-care provider) and deployed at a SHF in Anaheim, CA. Collaborations with local response agencies (Los Angeles, Orange County, San Bernardino, San Diego) and national entities (FEMA, Red Cross, NFPA/FPRF) will mesh needs of emergency responders with caregivers. CareDEX will be evaluated using diverse scenarios - a wildfire event triggering relocation, wildfires coupled with a pandemic, and rapid onset earthquake events with small warning times and increased uncertainty.
The CIVIC Innovation Challenge is a collaboration with Department of Energy, Department of Homeland Security Science and Technology Directorate, Federal Emergency Management Agency (FEMA), and the National Science Foundation