This project aims to enable cyber-physical systems that can be worn on the body in order to one day allow their users to touch, feel, and manipulate computationally simulated three-dimensional objects or digital data in physically realistic ways, using the whole hand. It will do this by precisely measuring touch and movement-induced displacements of the skin in the hand, and by reproducing these signals interactively, via new technologies to be developed in the project. The resulting systems will offer the potential to impact a wide range of human activities that depend on touch and interaction with the hands. The project seeks to enable new applications for wearable cyber physical interfaces that may have broad applications in health care, manufacturing, consumer electronics, and entertainment. Although human interactive technologies have advanced greatly, current systems employ only a fraction of the sensorimotor capabilities of their users, greatly limiting applications and usability. The development of whole-hand haptic interfaces that allow their wearers to feel and manipulate digital content has been a longstanding goal of engineering research, but has remained far from reality. The reason can be traced to the difficulty of reproducing or even characterizing the complex, action-dependent stimuli that give rise to touch sensations during everyday activities. This project will pioneer new methods for imaging complex haptic stimuli, consisting of movement dependent skin strain and contact-induced surface waves propagating in skin, and for modeling the dependence of these signals on hand kinematics during grasping. It will use the resulting fundamental advances to catalyze the development of novel wearable CPS, in the form of whole-hand haptic interfaces. The latter will employ surface wave and skin strain feedback to supply haptic feedback to the hand during interaction with real and computational objects, enabling a range of new applications in VR. The project will be executed through research in three main research areas. In the first, it will utilize novel contact and non-contact techniques based on data acquired through on-body sensor arrays to measure whole-hand mechanical stimuli and grasping kinematics at high spatial and temporal resolution. In a second research area, it will undertake data-driven systems modeling and analysis of statistical contingencies between the kinematic and cutaneous sensed during everyday activities. In a third research area, it will engineer and perceptually evaluate novel cyber physical systems consisting of haptic interfaces for whole hand interaction. In order to further advance the applications of these systems in medicine, through a collaboration with the Drexel College of Medicine, the project will develop new methods for assessing clinical skills of palpation during medical examination, with the aim of improving the efficacy of what is often the first, most common, and best opportunity for diagnosis, using physician's own sense of touch.

The project investigates a formal verification framework for artificial pancreas (AP) controllers that automate the delivery of insulin to patients with type-1 diabetes (T1D). AP controllers are safety critical: excessive insulin delivery can lead to serious, potentially fatal, consequences. The verification framework under development allows designers of AP controllers to check that their control algorithms will operate safely and reliably against large disturbances that include patient meals, physical activities, and sensor anomalies including noise, delays, and sensor attenuation. The intellectual merits of the project lie in the development of state-of-the-art formal verification tools, that reason over mathematical models of the closed-loop including external disturbances and insulin-glucose response. These tools perform an exhaustive exploration of the closed loop system behaviors, generating potentially adverse situations for the control algorithm under verification. In addition, automatic techniques are being investigated to help AP designers improve the control algorithm by tuning controller parameters to eliminate harmful behaviors and optimize performance. The broader significance and importance of the project are to minimize the manual testing effort for AP controllers, integrate formal tools in the certification process, and ultimately ensure the availability of safe and reliable devices to patients with type-1 diabetes. The framework is made available to researchers who are developing AP controllers to help them verify and iteratively improve their designs. The team is integrating the research into the educational mission by designing hands-on courses to train undergraduate students in the science of Cyber-Physical Systems (CPS) using the design of AP controllers as a motivating example. Furthermore, educational material that explains the basic ideas, current challenges and promises of the AP concept is being made available to a wide audience that includes patients with T1D, their families, interested students, and researchers. The research is being carried out collaboratively by teams of experts in formal verification for Cyber-Physical Systems, control system experts with experience designing AP controllers, mathematical modeling experts, and clinical experts who have clinically evaluated AP controllers. To enable the construction of the verification framework from the current state-of-the-art verification tools, the project is addressing major research challenges, including (a) building plausible mathematical models of disturbances from available clinical datasets characterizing human meals, activity patterns, and continuous glucose sensor anomalies. The resulting models are integrated in a formal verification framework; (b) simplifying existing models of insulin glucose response using smaller but more complex delay differential models; (c) automating the process of abstracting the controller implementation for the purposes of verification; (d) producing verification results that can be interpreted by control engineers and clinical researchers without necessarily understanding formal verification techniques; and (e) partially automating the process of design improvements to potentially eliminate severe faults and improve performance. The framework is evaluated on a set of promising AP controller designs that are currently under various stages of clinical evaluation.

More than one million people including many wounded warfighters from recent military missions are living with lower-limb amputation in the United States. This project will design wearable body area sensor systems for real-time measurement of amputee's energy expenditure and will develop computer algorithms for automatic lower-limb prosthesis optimization. The developed technology will offer a practical tool for the optimal prosthetic tuning that may maximally reduce amputee's energy expenditure during walking. Further, this project will develop user-control technology to support user's volitional control of lower-limb prostheses. The developed volitional control technology will allow the prosthesis to be adaptive to altered environments and situations such that amputees can walk as using their own biological limbs. An optimized prosthesis with user-control capability will increase equal force distribution on the intact and prosthetic limbs and decrease the risk of damage to the intact limb from the musculoskeletal imbalance or pathologies. Maintenance of health in these areas is essential for the amputee's quality of life and well-being. Student participation is supported. This research will advance Cyber-Physical Systems (CPS) science and engineering through the integration of sensor and computational technologies for the optimization and control of physical systems. This project will design body area sensor network systems which integrate spatiotemporal information from electromyography (EMG), electroencephalography (EEG) and inertia measurement unit (IMU) sensors, providing quantitative, real-time measurements of the user's physical load and mental effort for personalized prosthesis optimization. This project will design machine learning technology-based, automatic prosthesis parameter optimization technology to support in-home prosthesis optimization by users themselves. This project will also develop an EEG-based, embedded computing-supported volitional control technology to support user?s volitional control of a prosthesis in real-time by their thoughts to cope with altered situations and environments. The technical advances from this project will provide wearable and wireless body area sensing solutions for broader applications in healthcare and human-CPS interaction applications. The explored computational methods will be broadly applicable for real-time, automatic target recognition from spatiotemporal, multivariate data in CPS-related communication and control applications. This synergic project will be implemented under multidisciplinary team collaboration among computer scientists and engineers, clinicians and prosthetic industry engineers. This project will also provide interdisciplinary, CPS relevant training for both undergraduate and graduate students by integrating computational methods with sensor network, embedded processors, human physical and mental activity recognition, and prosthetic control.

Stroke is the leading cause of long-term disability in the US with approximately 7 million stroke survivors living in the US today and for patients with neurological disorders, it has been shown that limited gait velocity commonly results in walking that is predominantly restricted to the household. Unlike traditional exoskeletons which contain rigid linkage elements, the vision for this work is for exosuits that use soft materials such as textiles to provide a more conformal, unobtrusive and compliant means to interface to the human body. This represents a fundamental change in the paradigm of how people have viewed and designed wearable robots for the last half a century. Such a solution would have broad impact beyond the stroke patient population and could provide benefit to children with Cerebral Palsy or elderly individuals with muscle weakness. In addition there are plans to create a set of novel instructional educational toolkits for patient-in-the-loop cyber-physical systems that will be shared via an online portal and the CPS Virtual Organization (CPS-VO). With a patient-in-the-loop CPS, the patient, the physical suit, the computational control algorithms and the task/environment form a system in which all of the elements need to seamlessly interact. Through a modeling and experimental approach involving extensive human subjects studies, the team aims to create a unified engineering, biomechanical and physiological framework for designing and evaluating patient-in-the-loop CPS that include co-operative controllers that adapt in real-time to the patient to ensure safety and reliability an integrated system. Specifically the project will seek to gain a fundamental understanding of how to (1) analytically and experimentally characterize how forces are transmitted from these soft systems to the patient through the underlying soft tissue so as to generate assistance, (2) apply the optimal magnitude and timing of assistance to the patient to promote a more symmetric and natural gait by monitoring biomechanical, physiological and suit sensor data and (3) fuse information from different sensors monitoring patient motion and interaction forces to create an integrated CPS with a co-operative controller than can adapt to non-periodic movements of the patient.

This project represents a cross-disciplinary collaborative research effort on developing rigorous, closed-loop approaches for designing, simulating, and verifying medical devices. The work will open fundamental new approaches for radically accelerating the pace of medical device innovation, especially in the sphere of cardiac-device design. Specific attention will be devoted to developing advanced formal methods-based approaches for analyzing controller designs for safety and effectiveness; and devising methods for expediting regulatory and other third-party reviews of device designs. The project team includes members with research backgrounds in computer science, electrical engineering, biophysics, and cardiology; the PIs will use a coordinated approach that balances theoretical, experimental and practical concerns to yield results that are intended to transform the practice of device design while also facilitating the translation of new cardiac therapies into practice. The proposed effort will lead to significant advances in the state of the art for system verification and cardiac therapies based on the use of formal methods and closed-loop control and verification. The animating vision for the work is to enable the development of a true in silico design methodology for medical devices that can be used to speed the development of new devices and to provide greater assurance that their behaviors match designers' intentions, and to pass regulatory muster more quickly so that they can be used on patients needing their care. The scientific work being proposed will serve this vision by providing mathematically robust techniques for analyzing and verifying the behavior of medical devices, for modeling and simulating heart dynamics, and for conducting closed-loop verification of proposed therapeutic approaches. The acceleration in medical device innovation achievable as a result of the proposed research will also have long-term and sustained societal benefits, as better diagnostic and therapeutic technologies enter into the practice of medicine more quickly. It will also yield a collection of tools and techniques that will be applicable in the design of other types of devices. Finally, it will contribute to the development of human resources and the further inclusion of under-represented groups via its extensive education and outreach programs, including intensive workshop experiences for undergraduates

This project represents a cross-disciplinary collaborative research effort on developing rigorous, closed-loop approaches for designing, simulating, and verifying medical devices. The work will open fundamental new approaches for radically accelerating the pace of medical device innovation, especially in the sphere of cardiac-device design. Specific attention will be devoted to developing advanced formal methods-based approaches for analyzing controller designs for safety and effectiveness; and devising methods for expediting regulatory and other third-party reviews of device designs. The project team includes members with research backgrounds in computer science, electrical engineering, biophysics, and cardiology; the PIs will use a coordinated approach that balances theoretical, experimental and practical concerns to yield results that are intended to transform the practice of device design while also facilitating the translation of new cardiac therapies into practice. The proposed effort will lead to significant advances in the state of the art for system verification and cardiac therapies based on the use of formal methods and closed-loop control and verification. The animating vision for the work is to enable the development of a true in silico design methodology for medical devices that can be used to speed the development of new devices and to provide greater assurance that their behaviors match designers' intentions, and to pass regulatory muster more quickly so that they can be used on patients needing their care. The scientific work being proposed will serve this vision by providing mathematically robust techniques for analyzing and verifying the behavior of medical devices, for modeling and simulating heart dynamics, and for conducting closed-loop verification of proposed therapeutic approaches. The acceleration in medical device innovation achievable as a result of the proposed research will also have long-term and sustained societal benefits, as better diagnostic and therapeutic technologies enter into the practice of medicine more quickly. It will also yield a collection of tools and techniques that will be applicable in the design of other types of devices. Finally, it will contribute to the development of human resources and the further inclusion of under-represented groups via its extensive education and outreach programs, including intensive workshop experiences for undergraduates.

This NSF-FDA Scholar-In-Residence award supports translational research in modeling to inform future medical device design and approval processes. It is supported by the NSF Cyber-Physical Systems program in the Division of Computer and Network Systems in the Directorate for Computer and Information Science and Engineering. Sudden cardiac death is the leading cause of fatalities in the industrialized world. One in five people in the United States is affected by some sort of heart disease and one third of all deaths are due to cardiac diseases with an economic impact of about $200 billion a year. Most of these deaths result from arrhythmias, particularly fibrillation, which is rapid, disorganized electrical activity. The classification of arrhythmias as either reentrant or focal is of clinical significance, yet is difficult to assess. The FDA is responsible for regulating the systems and algorithms that aim to make this important differentiation. Such differentiation is a complex task involving the analysis of complex spatio-temporal patterns of electrical activity. The objectives of this project are to identify the key features of fibrillation that models should represent, to compare how well (or poorly) existing models correspond to measured values of these features, and to develop models that better represent fibrillation. The project develops and extends cell and tissue models and explores the analysis of clinical, experimental and simulation data from the perspective of regulatory science at the FDA, including verification, validation, and uncertainty quantification (VVUQ). The project seeks to 1) validate and create new models that reproduce not only single-cell dynamics, but also experimental and clinically relevant physiological dynamics in tissue and 2) initiate a new developmental framework that the FDA can use not only to test cardiac electrophysiology devices but also to characterize and verify massive submissions of therapeutic compounds obtained by computer-aided drug design methods. The research is conducted in collaboration with the Center for Devices and Radiological Health at FDA, and is aimed at developing tools that can characterize and evaluate real-world performance of devices. This will help the FDA to better regulate and verify the safety and effectiveness of devices that are developed to treat and terminate cardiac arrhythmias. All results from this project will be made freely available to the research community and to the general public.

The problem of controlling biomechatronic systems, such as multiarticulating prosthetic hands, involves unique challenges in the science and engineering of Cyber Physical Systems (CPS), requiring integration between computational systems for recognizing human functional activity and intent and controlling prosthetic devices to interact with the physical world. Research on this problem has been limited by the difficulties in noninvasively acquiring robust biosignals that allow intuitive and reliable control of multiple degrees of freedom (DoF). The objective of this research is to investigate a new sensing paradigm based on ultrasonic imaging of dynamic muscle activity. The synergistic research plan will integrate novel imaging technologies, new computational methods for activity recognition and learning, and high-performance embedded computing to enable robust and intuitive control of dexterous prosthetic hands with multiple DoF. The interdisciplinary research team involves collaboration between biomedical engineers, electrical engineers and computer scientists. The specific aims are to: (1) research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity (2) develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor and (3) perform experiments to evaluate the real-time control of a prosthetic hand. The proposed research methods are broadly applicable to assistive technologies where physical systems, computational frameworks and low-power embedded computing serve to augment human activities or to replace lost functionality. The research will advance CPS science and engineering through integration of portable sensors for image-based sensing of complex adaptive physical phenomena such as dynamic neuromuscular activity, and real-time sophisticated image understanding algorithms to interpret such phenomena running on low-power high performance embedded systems. The technological advances would enable practical wearable image-based biosensing, with applications in healthcare, and the computational methods would be broadly applicable to problems involving activity recognition from spatiotemporal image data, such as surveillance. This research will have societal impacts as well as train students in interdisciplinary methods relevant to CPS. About 1.6 million Americans live with amputations that significantly affect activities of daily living. The proposed project has the long-term potential to significantly improve functionality of upper extremity prostheses, improve quality of life of amputees, and increase the acceptance of prosthetic limbs. This research could also facilitate intelligent assistive devices for more targeted neurorehabilitation of stroke victims. This project will provide immersive interdisciplinary CPS-relevant training for graduate and undergraduate students to integrate computational methods with imaging, processor architectures, human functional activity and artificial devices for solving challenging public health problems. A strong emphasis will be placed on involving undergraduate students in research as part of structured programs at our institution. The research team will involve students with disabilities in research activities by leveraging an ongoing NSF-funded project. Bioengineering training activities will be part of a newly developed undergraduate curriculum and a graduate curriculum under development. The synergistic research plan has been designed to advance CPS science and engineering through the development of new computational methods for dynamic activity recognition and learning from image sequences, development of novel wearable imaging technologies including high-performance embedded computing, and real-time control of a physical system. The specific aims are to: (1) Research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity. The first aim has three subtasks designed to collect, analyze and understand image sequences associated with functional tasks. (2) Develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor. The second aim has two subtasks designed to integrate wearable imaging sensors with a real-time computational platform. (3) Perform experiments to evaluate the real-time control of a prosthetic hand. The third aim will integrate the wearable image acquisition system developed in Aim 2, and the image understanding algorithms developed in Aim 1, for real-time evaluation of the control of a prosthetic hand interacting with a virtual reality environment. Successful completion of these aims will result in a real-time system that acquires image data from complex neuromuscular activity, decodes activity intent from spatiotemporal image data using computational algorithms, and controls a prosthetic limb in a virtual reality environment in real time. Once developed and validated, this system can be the starting point for developing a new class of sophisticated control algorithms for intuitive control of advanced prosthetic limbs, new assistive technologies for neurorehabilitation, and wearable real-time imaging systems for smart health applications.

In telerobotic applications, human operators interact with robots through a computer network. This project is developing tools to prevent security threats in telerobotics, by monitoring and detecting malicious activities and correcting for them. To develop tools to prevent and mitigate security threats against telerobotic systems, this project adapts cybersecurity methods and extends them to cyber-physical systems. Knowledge about physical constraints and interactions between the cyber and physical components of the system are leveraged for security. A monitoring system is developed which collects operator commands and robot feedback information to perform real-time verification of the operator. Timely and reliable detection of any discrepancy between real and spoofed operator movements enables quick detection of adversarial activities. The results are evaluated on the UW-developed RAVEN surgical robot. This project brings together research in robotics, computer and network security, control theory and machine learning, in order to gain better understanding of complex teleoperated robotic systems and to engineer telerobotic systems that provide strict safety, security and privacy guarantees. The results are relevant and applicable to a wide range of applications, including telerobotic surgery, search and rescue missions, military operations, underwater infrastructure and repair, cleanup and repair in hazardous environments, mining, as well as manipulation/inspections of objects in low earth orbit. The project algorithms, software and hardware are being made available to the non-profit cyber-physical research community. Graduate and undergraduate students are being trained in cyber-physical systems security topics, and K-12, community college students and under-represented minority students are being engaged.

This project investigates a new type of cyber-physical system (CPS), comprising magnetic nanoparticles in a fluidic environment such as human tissue whose motion is controlled by a computer via a magnetic field. The research aims to develop computational and experimental tools to perform the dynamic modeling, closed loop control and experimental validation of such a system of nanoparticles under guidance and observation using a magnetic resonance imaging (MRI) environment. The envisioned CPS infrastructure is composed of a new computational platform to perform 3D simulation, visualization and post-processing analysis of the aggregation and disaggregation process of magnetic nanoparticles within a fluidic environment like the small arteries and arterioles or fluid-filled cavities of the human body. It also includes the development of robust control algorithms for the guidance of a swarm of magnetic nanoparticles in a MRI environment. Experimental validation is to be performed in clinical MRI scanners and in customized laboratory test-beds that generate controllable magnetic fields able to move magnetic nanoparticles in fluidic environments. Potential applications of this basic research include nano-robotic drug delivery systems, composed of a system of magnetic nanoparticles guided by MRI scanners for targeted drug delivery in the human body. The project integrates education through participation of graduate and undergraduate students in the research, and involvement of the PI and graduate students in several outreach activities for students in high and middle schools.