Abstract
Complex surgical procedures in hospitals are increasingly aided by robotic surgery systems, often at the request of patients. These systems allow greatly increased precision, reach and flexibility to the surgeon. However, their powerful capabilities entail substantial system complexity in both hardware and software. The high probability of serious injuries should a malfunction occur calls for rigorous assessment and monitoring of the reliability and safety of these cyber-physical systems. In this research project, a framework for assessing and monitoring the reliability and safety of robotic surgery systems during development, field testing, and general deployment is being developed. The proposed framework complements existing techniques used in earlier phases of validation by taking into account how surgeons actually use a robotic surgery system, how it is affected by operating conditions, and how its observable behavior is related to its hardware and software dynamics. Before deployment, this framework uses accurate simulations to assess pre-clinical reliability. After deployment, the framework uses data collection through online monitoring of the system as it is being used in the field, followed by analysis to obtain assessments of operational reliability and safety. The collected data is also used to improve the simulations for future testing. The framework also aims to support post-market surveillance of these systems by providing a workable basis for reassessing reliability and safety properties after system maintenance. The developed tools and methods will also have applications in the validation of safety and reliability of other medical devices with embedded software and other cyber-physical systems in general.
M. Cenk Cavusoglu
Performance Period: 09/01/2010 - 12/31/2015
Institution: Case Western Reserve University
Sponsor: National Science Foundation
Award Number: 1035602
Abstract
The objective of this research is to develop a cyber-physical system capable of displaying the in vivo surgical area directly onto patients' skin in real-time high definition. This system will give surgeons an x-ray vision experience, since they see directly through the skin, and remove a spatial bottleneck and additional scarring caused by laparoscopes in minimally invasive surgery. The approach is to develop micro-cameras that: occupy no space required by surgical tools, produce no additional scarring to the patient, and transfer wireless high-definition video images. A virtual view generating system will project the panoramic videos from all cameras to the right spot on the patient?s body with geometry and color distortion compensation. A surgeon-camera-interaction system will be investigated to allow surgeons to control viewpoint with gesture recognition and finger tracking. Novel techniques will be developed for zero-latency high-definition wireless video transfer through the in vivo/ex vivo medium. Image viewpoint alignment and distortion compensation in real time will also be investigated. The results will be a potential paradigm shift in minimally invasive surgery. The proposed work benefits the millions of surgeries capable of being performed through a single incision in the abdomen by providing virtually transparent skin to surgeons who will enjoy all the visual benefits of open-cavity surgery without all the associated risks to the patient. The goals of this research are extremely hands-on and immediately applicable to outreach activities that can excite youth, minority students, and others about the science, medicine a and engineering careers.
Yu Sun
<p><span style="color: rgb(121, 121, 121); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">I am currently an Assistant Professor in the Department of Computer Science and Engineering at the University of South Florida. I was a Postdoctoral Associate at </span><a href="http://www.merl.com/" style="margin: 0px; padding: 0px; color: rgb(82, 82, 82); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">Mitsubishi Electric Research Laboratories (MERL)</a><span style="color: rgb(121, 121, 121); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">, Cambridge, MA from Dec. 2007 to May 2008 and a Postdoctoral Associate in the School of Computing at the University of Utah from May 2008 to May 2009. I received my B.S. and M.S. degrees in electrical engineering from Dalian University of Technology, Dalian, China, in 1997 and 2000, respectively, and Ph.D. degree in computer science from the University of Utah, Salt Lake City, in 2007. </span><br style="margin: 0px; padding: 0px; color: rgb(121, 121, 121); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">
<br style="margin: 0px; padding: 0px; color: rgb(121, 121, 121); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">
<span style="color: rgb(121, 121, 121); font-family: 'Trebuchet MS', Arial, Helvetica, sans-serif; font-size: medium; line-height: 22px; text-align: -webkit-left; ">My research interests include robotics, haptics, computer vision, human computer interaction (HCI), and medical applications.</span></p>
Performance Period: 09/15/2010 - 08/31/2015
Institution: University of South Florida
Sponsor: National Science Foundation
Award Number: 1035594
Abstract
The objective of this research is to apply grammatical inference models recently developed in the field of linguistics and phonology, as a basis for abstraction, composition, symbolic control, and learning in distributed multi-agent cyber-physical systems. The approach is to map the system dynamics, specifications, and task interdependences to finite abstract models, and then describe the desired behavior of the system in an appropriate grammar that can be decomposed into local agent specifications. In this framework, the agents can learn the behavior of their environment by observing its dynamics, and update their specifications accordingly. The proposed approach to learning in cyber-physical systems, which is based on grammatical inference at a purely discrete level, is a significant departure from current works. Following this approach, one can reason about large-scale processes resulting from event interdependencies between agents, without having to construct large product systems. To realize this plan, specific technical advances on modeling, abstraction, and control synthesis are proposed. Questions related to formally factoring and composing heterogeneous systems are pervasive in the fields of formal languages and computational learning. There are also applications of commercial significance in the area of discovering new azeotropic mixtures based on documented pairs of compounds that are known to have the particular property. Proposed dissemination and outreach activities include the involvement of middle and high school students and teachers, integrated in existing NSF-sponsored programs at the University of Delaware and Boston University.
Calin Belta
Performance Period: 09/15/2010 - 08/31/2015
Institution: Trustees of Boston University
Sponsor: National Science Foundation
Award Number: 1035588
Abstract
The objective of this research is to apply grammatical inference models recently developed in the field of linguistics and phonology, as a basis for abstraction, composition, symbolic control, and learning in distributed multi-agent cyber-physical systems. The approach is to map the system dynamics, specifications, and task interdependences to finite abstract models, and then describe the desired behavior of the system in an appropriate grammar that can be decomposed into local agent specifications. In this framework, the agents can learn the behavior of their environment by observing its dynamics, and update their specifications accordingly. The proposed approach to learning in cyber-physical systems, which is based on grammatical inference at a purely discrete level, is a significant departure from current works. Following this approach, one can reason about large-scale processes resulting from event interdependencies between agents, without having to construct large product systems. To realize this plan, specific technical advances on modeling, abstraction, and control synthesis are proposed. Questions related to formally factoring and composing heterogeneous systems are pervasive in the fields of formal languages and computational learning. There are also applications of commercial significance in the area of discovering new azeotropic mixtures based on documented pairs of compounds that are known to have the particular property. Proposed dissemination and outreach activities include the involvement of middle and high school students and teachers, integrated in existing NSF-sponsored programs at the University of Delaware and Boston University.
Herbert Tanner
Performance Period: 09/15/2010 - 08/31/2015
Institution: University of Delaware
Sponsor: National Science Foundation
Award Number: 1035577
Abstract
The objective of this research is to enable cyberphysical systems (CPS) to be context-aware of people in the environment and to use data from real-world probabilistic sensors. The approach is (1) to use radio tomography (RT) and RFID to provide awareness (location and potential identification) of every person in a building or area, and (2) to develop new middleware tools to enable context-aware computing systems to use probabilistic data, thus allowing new applications to exploit sometimes unreliable estimates of the environment.The intellectual merit of the proposal is in the development of new algorithms and models for building-scale RT with low radio densities and across multiple frequencies; the development of efficient multichannel access protocols for rapid and adaptive peer-to-peer measurements; the development of space-time and probabilistic data representations for use in stream-based context awareness systems and for merging ID and non-ID data; (4) and the development of a human context-aware software development toolkit that interfaces between probabilistic data and context-aware applications.
The proposal impacts broadly the area of Cyber-physical systems that reason about human presence and rely on noisy and potentially ambiguous (practical) sensors. The research has additional dramatic impact in: (1) smart facilities which automatically enforce safety, privacy, and security procedures, increasing the ability to respond in emergency situations and prevent accidents and sabotage; (2) elder care, to monitor for physical or social decline so that effective intervention can be implemented, extending the period elders can live in their own home, without pervasive video surveillance.
Neal Patwari
Performance Period: 09/15/2010 - 08/31/2015
Institution: University of Utah
Sponsor: National Science Foundation
Award Number: 1035565
Abstract
The objective of this research is to develop an atomic force microscope based cyber-physical system that can enable automated, robust and efficient assembly of nanoscale components such as nanoparticles, carbon nanotubes, nanowires and DNAs into nanodevices. The approach in this project is based on the premise that automated, robust and efficient nanoassembly can be achieved through tip based pushing in an atomic force microscope with intermittent local scanning of nanoscale components. In particular, in order to resolve temporally and spatially continuous movement of nanoscale components under tip pushing, the research is exploring the combination of intermittent local scanning and interval non-uniform rational B-spline based isogeometric analysis in this research. Successful completion of this research is expected to lead to foundational theories and algorithmic infrastructures for effective integration of physical operations (pushing and scanning) and computation (planning and simulation) for robust, efficient and automated nanoassembly. The resulting theories and algorithms will also be applicable to a broader set of cyber physical systems. If successful, this research will lead to leap progress in nanoscale assembly, from prototype demonstration to large-scale manufacturing. Through its integrated research, education and outreach activities, this project is providing experiences and understanding in cyber-physical systems and nanoassembly for students from high schools to graduate schools. The goal is to increase interest in science and engineering among domestic students and therefore strengthen our competitiveness in the global workforce.
Guangyong Li
Performance Period: 09/15/2010 - 08/31/2013
Institution: University of Pittsburgh
Sponsor: National Science Foundation
Award Number: 1035563
Abstract
The objective of this research is to develop advanced distributed monitoring and control systems for civil infrastructure. The approach uses a cyber-physical co-design of wireless sensor-actuator networks and structural monitoring and control algorithms. The unified cyber-physical system architecture and abstractions employ reusable middleware services to develop hierarchical structural monitoring and control systems. The intellectual merit of this multi-disciplinary research includes (1) a unified middleware architecture and abstractions for hierarchical sensing and control; (2) a reusable middleware service library for hierarchical structural monitoring and control; (3) customizable time synchronization and synchronized sensing routines; (4) a holistic energy management scheme that maps structural monitoring and control onto a distributed wireless sensor-actuator architecture; (5) dynamic sensor and actuator activation strategies to optimize for the requirements of monitoring, computing, and control; and (6) deployment and empirical validation of structural health monitoring and control systems on representative lab structures and in-service multi-span bridges. While the system constitutes a case study, it will enable the development of general principles that would be applicable to a broad range of engineering cyber-physical systems. This research will result in a reduction in the lifecycle costs and risks related to our civil infrastructure. The multi-disciplinary team will disseminate results throughout the international research community through open-source software and sensor board hardware. Education and outreach activities will be held in conjunction with the Asia-Pacific Summer School in Smart Structures Technology jointly hosted by the US, Japan, China, and Korea.
Gul Agha
Performance Period: 10/01/2010 - 09/30/2014
Institution: University of Illinois at Urbana-Champaign
Sponsor: National Science Foundation
Award Number: 1035562
Abstract
The objective of this research is the development of methods and software that will allow robots to detect and localize objects using Active Vision and develop descriptions of their visual appearance in terms of shape primitives. The approach is bio inspired and consists of three novel components. First, the robot will actively search the space of interest using an attention mechanism consisting of filters tuned to the appearance of objects. Second, an anthropomorphic segmentation mechanism will be used. The robot will fixate at a point within the attended area and segment the surface containing the fixation point, using contours and depth information from motion and stereo. Finally, a description of the segmented object, in terms of the contours of its visible surfaces and a qualitative description of their 3D shape will be developed. The intellectual merit of the proposed approach comes from the bio-inspired design and the interaction of visual learning with advanced behavior. The availability of filters will allow the triggering of contextual models that work in a top-down fashion meeting at some point the bottom-up low-level processes. Thus, the approach defines, for the first time, the meeting point where perception happens. The broader impacts of the proposed effort stem from the general usability of the proposed components. Adding top-down attention and segmentation capabilities to robots that can navigate and manipulate, will enable many technologies, for example household robots or assistive robots for the care of the elders, or robots in manufacturing, space exploration and education.
Yiannis Aloimonos
Performance Period: 09/15/2010 - 08/31/2013
Institution: University of Maryland College Park
Sponsor: National Science Foundation
Award Number: 1035542
Abstract
The objective of the research is to develop tools for comprehensive design and optimization of air traffic flow management capabilities at multiple spatial and temporal resolutions: a national airspace-wide scale and one-day time horizon (strategic time-frame); and at a regional scale (of one or a few Centers) and a two-hour time horizon (tactical time-frame). The approach is to develop a suite of tools for designing complex multi-scale dynamical networks, and in turn to use these tools to comprehensively address the strategic-to-tactical traffic flow management problem. The two directions in tool development include 1) the meshed modeling/design of flow- and queueing-networks under network topology variation for cyber- and physical- resource allocation, and 2) large-scale network simulation and numerical analysis. This research will yield aggregate modeling, management design, and validation tools for multi-scale dynamical infrastructure networks, and comprehensive solutions for national-wide strategic-to-tactical traffic flow management using these tools. The broader impact of the research lies in the significant improvement in cost and equity that may be achieved by the National Airspace System customers, and in the introduction of systematic tools for infrastructure-network design that will have impact not only in transportation but in fields such as electric power network control and health-infrastructure design. The development of an Infrastructure Network Ideas Cluster will enhance inter-disciplinary collaboration on the project topics and discussion of their potential societal impact. Activities of the cluster include cross-university undergraduate research training, seminars on technological and societal-impact aspects of the project, and new course development.
Dengfeng Sun
Performance Period: 09/01/2010 - 08/31/2014
Institution: Purdue University
Sponsor: National Science Foundation
Award Number: 1035532
Abstract
The objective of this research is the creation of a coastal observing system that enables dense, in situ, 4D sensing through networked, sensor-equipped underwater drifters. The approach is to develop the technologies required to deploy a swarm of autonomous buoyancy controlled drifters, which are vehicles that can control their depth, but are otherwise carried entirely by the ocean currents. Such Lagrangian sampling promises to deliver a wealth of new data, ranging from applications in physical oceanography (mapping 3D currents), biology (observing the dispersion of larvae and nutrients), environmental science (tracking coastal pollutants and effluents from storm drains), and security (monitoring harbors and ports). This observing system fundamentally requires accurate positions of the drifters (to interpret the spatial correlations of data samples), swarm control algorithms (to achieve desired sampling topologies), and wireless communication (to coordinate between the individual drifters). This research will create distributed techniques to self-localize the drifter swarm, novel swarm control algorithms that enable topology manipulation while purely leveraging the stratified flow environment, and efficient wireless underwater communication for information sharing. This project has significant societal impact and educational elements. Underwater drifter swarms will enable novel insights into a wide array of scientific questions, including understanding plankton transport, accumulation and dispersion as well as monitoring harmful algal blooms. Undergraduates will play an active role in many aspects of this project, thereby offering them a uniquely interdisciplinary experience. Finally, outreach to high school students will occur through the UCSD COSMOS summer program.
Jules Jaffe
Performance Period: 09/15/2010 - 08/31/2014
Institution: University of California-San Diego Scripps Institute of Oceanography
Sponsor: National Science Foundation
Award Number: 1035518