Since 2000, surgical robots have been used in over 1.75 million minimally invasive procedures in the U.S. across various surgical specialties, including gynecological, urological, general, cardiothoracic, and head and neck surgery. Robotic surgical procedures promise decreased complication rates and morbidity, due to the minimally invasive nature of the procedures. A detailed analysis (also reported to the FDA) of the adverse events associated with the surgical robot indicates that despite the increased number of robotic procedures and their greater utilization, the rate of adverse events has remained relatively steady over the last 14 years. Even though current surgical robots are designed with safety mechanisms in mind, in practice several significant challenges exist in enabling timely and accurate detection and mitigation of adverse incidents during surgery. Toward this goal, the project will address (i) an in-depth analysis of incident causes, which takes into account the interactions among the system components, human operators, and patients; (ii) resiliency assessment of the robotic systems in the presence of realistic safety hazards, reliability failures, and malicious tampering; and (iii) continuous monitoring for detection of safety, reliability, and security violations to ensure patient safety. The intellectual merit of this work lies in: (i) systems-theoretic approach driven by real data on safety hazards and medical equipment recalls, to identify causes leading to violation of safety constraints at different layers of the cyber and physical system-control-structure; (ii) creation of a unique safety hazard simulation engine to perform injections into robot control software and emulate realistic safety hazard scenarios in a virtual environment; (iii) an adaptive method for rapid detection of events that lead to safety violations, based on continuous monitoring of human operator actions, robot state, and patient status, in conjunction with a probabilistic graph-model that captures dependencies between the causal factors leading to safety hazards; and (iv) experimental validation using the real robot to assess monitoring and protection mechanisms in the presence of realistic safety hazards, reliability faults, and security exploits (recreated using safety hazard simulation engine). The broader impact of the project is a methodology for design and resiliency assessment of a larger class of control cyber-physical systems, which involve humans in the on-line decision making loop. Application of the methodology to robot-assisted surgery demonstrates the strength and practicality of the approach and is likely to attract interest from areas of academia and industry in which cyber-physical systems are either a subject of study or the basis for delivering a service (e.g., transportation or electric power grids). This project's educational outreach encompasses strategies for broadening participation in multi-disciplinary projects spanning medicine and engineering.

Pedestrian safety continues to be a significant concern in urban communities. Several recent reports indicate that injuries and fatalities in pedestrian-related accidents are steadily rising and that pedestrian distraction is one of the leading causes in such accidents. Existing systems and techniques for improving pedestrian safety, which primarily operate on users' smartphones and mobile devices in a stand-alone fashion, have several design drawbacks and performance and usability concerns that have precluded their successful adoption and usage. The goal of this project is to improve pedestrian safety by designing accurate, efficient and usable tools and techniques, which can be easily adopted by urban users. In order to accomplish this goal, this project plans to pursue a focused research agenda involving novel technologies and several exploratory and untested ideas. As part of the proposed pedestrian safety framework, accurate and energy-efficient on-device distraction detection techniques will be developed by employing multi-sensor and heterogeneous data available from upcoming mobile and wearable devices. In this direction, supervised and semi-supervised learning will be used to design efficient activity classification and distraction prediction techniques which will be empirically evaluated using proof-of-concept implementations. Unlike existing stand-alone approaches, the proposed framework employs a connected-community approach to accurately capture the impact of both a pedestrian's own actions, as well as the actions of others, on his/her safety. This involves the design and implementation of a privacy-preserving and cloud-assisted data-analytics engine to capture, analyze and notify pedestrians of impending hazardous situations from the crowd-sourced distraction data obtained from participating users. Finally, a comprehensive performance and usability evaluation will be conducted by deploying a large-scale testbed involving participants from Wichita State University's (WSU) campus community. The project outcomes, including the planned testbed, will have a significant impact on improving pedestrian safety within the WSU campus community. If successful, similar trials at an urban or city-wide scale can also be envisioned. In addition to improving pedestrian safety, this project will educate users and participants on the impact of technology on pedestrian safety and its role in improving the same. Project outcomes and results will be disseminated by means of peer-reviewed publications, white papers and open-source applications. Applications and anonymous data collected from the planned testbed will be appropriately disseminated to facilitate additional research and advances in the area of pedestrian safety technology.

Cyberphysical (CPS) systems are set to become ever faster, driven by technological advances that push them toward speed limits set by fundamental physics. This proposal addresses the need for a theory of the dynamical behavior of CPS systems in the sub-second regime beyond human intervention times. Ultrafast instabilities have already been observed in such systems. The theory will allow for networking at multiple scales, coupling across multiple temporal and spatial scales, imperfect network communications and sensors, as well as adaptive reorganization and reconfiguration of the system. Theoretical findings will be checked against available empirical data, e.g., from the decentralized network of autonomous market exchanges with its mandated sensor systems. The project will inform the extent to which instabilities can build up across timescales, potentially threatening CPS system stability on a global level. The project goal is a theoretical description of the dynamics in decentralized networks of semi-autonomous machines in which an ecology of algorithms, sensors and network links may be operating, adapting and even competing in response to external inputs. Attention will be paid to the regime of sub-second behavior where human intervention becomes impossible in real-time. The availability of data from such a system provides a test-bed for the multi-agent, complex network analyses to be developed. The project will address how instabilities can be mitigated and eventually controlled. The results are set to advance understanding of CPS system dynamics, not only among academics but also practitioners and regulatory bodies. Application areas that pervade modern life include market exchange systems, resource-allocation systems and remote sensing systems. Opportunities exist to integrate research and education concerning CPS applications across graduate and undergraduate classrooms, outreach through publications, and participation in K-12 activities.

Machining is a fundamental manufacturing capability critical to the production of end-user goods and systems, as well as the tooling and equipment used in virtually every industrial process. Machine tool programming to support these processes is critical for both production and cost estimation. However, currently available automated process planning methods constrain the tool to follow geometrically simple paths to minimize computational requirements, limit the tool velocity and/or tool orientation during a simple path to constant values, also to save computational burden, and/or process one geometrical feature at a time without attempting to optimize the entire process. This award supports fundamental research to provide knowledge needed for development of a novel computer-aided process planning and control architecture for integrated complex tool path generation and optimization. The resulting new mathematical algorithms will drive effective real-time control and optimization of manufacturing processes and enable substantial increases in machine productivity. These capabilities will have potential for broad-ranging impact on the domestic economy, which uses machining in the vast majority of products, due to its flexibility, speed, cost, and accuracy advantages relative to other processes. The research integrates several complementary technical domains, including advanced manufacturing, geometric computing, and high performance parallel computing. Conventional computer-aided manufacturing approaches for toolpath optimization are inherently post-hoc methods that are not well integrated for supporting simultaneous generation and optimization of the process plan. This research will investigate generic optimization and control architectures for automated toolpath optimization, which require global solutions to highly non-linear, constrained optimization problems in high-dimensional search spaces of tool motions with time-varying positions and orientations. To address this challenge, the research approach will utilize a bi-directional optimization scheme that consists of: (l) a top-down, multi-level decomposition of the problem into fundamental model motions (e.g., curved blocks, peel layers, tool swipes) and (2) a bottom-up optimization of these fundamental geometric model motions. The latter schema element will fully support the option of selecting cutting tools to maximize material removal rate and/or surface finish and will build upon theoretical formulations and efficient computing implementations of volume preserving offsetting, steady motion interpolation, and ball morphing surface interpolation. The resulting fundamental motion models and associated fast, parallel computing algorithms will provide for rapid analysis of swept regions, collision avoidance and computing material removal rates that are critical for facilitating rapid toolpath optimization.

The Visualization for Terrestrial and Aquatic Systems project helps environmental scientists produce visualizations for their own research and for presentation to other scientists and stakeholders including decision makers. A critical finding of work to date is the extent to which scientists use visualizations not only to explore data in new ways and present results, but also to work with stakeholders to jointly produce information that can be used during decision-making processes. The role of scientific visualization in the co-production of knowledge is as yet untested, even though this involvement could be critical in creating acceptable solutions, information, or technology. This proposal recasts VISTAS to co-produce visualization tools, i.e., exploring how negotiations between the users? needs and technological capacity shape the type of visualizations used and tools implemented, change or modify the research questions posed by scientists, and impact how results are interpreted so communities can respond to critical ecological challenges, including climate change. This is a unique experiment and collaboration among social-, computer-, and environmental scientists, with non-scientist stakeholders, to co-produce data visualizations for use in decision making. Social science methods will be used to explore knowledge co-production coupled with technology innovations that lead to community decision making to solve problems of climate change adaptation. The extent to which distinctions between scientific visualization for scientists and non-scientists need to be made will be determined, and unique visualizations will be developed jointly with project collaborators. The goal is to determine the influence of visualization on the co-production of knowledge among scientists and stakeholders on critical decisions related to climate adaptation. This project involves both computer scientists and social scientists. Computer science: VISTAS, a C++ scientific visualization application with significant GPU processing, helps environmental scientists produce images that allow them to ?see? the effects of topography on ecological phenomena. For this award, new visualization techniques will be developed, visualization and visual analytics research that enables effective presentations to decision makers will be conducted, and technical support for environmental- and social scientists will be provided. If time and funds permit extensions to the current software that render it both more usable by primary and secondary users, and more maintainable and extensible directly by primary users will be provided: VISTAS engineers will proceed with a longer term strategy of migrating from C++ to Python, which will enable more effective and flexible user interface development, end user programming of data or visualization plug-ins, and use of emerging and existing Python and R libraries for visual analytics. The social science inquiry will help determine how the co-production enables usable software that answers the needs of both environmental scientists who generate large difficult to interpret data sets as well as decision-makers who must balance multiple demands as they make important choices. Case studies with three collaborators will be conducted as they work with stakeholders to co-develop usable information; these are structured through a comparative pre/post-test design with three phases to explore changes in how participants view and communicate scientific results before and after involvement in visualization development. In the baseline phase VISTAS social scientists will work with participants to document their current understanding of their data, expectations for the visualization and analytic products, and ability and tools used to communicate science to others including non-scientists. During the development phase case participants will be observed as they work together to create the visualization and analytic products. The post-assessment phase seeks to determine changes in understanding of data and ability to communicate science as a result of participation in visualization development. The usability of different types of visualizations and analytic tools, identifying the characteristics that contribute to or distract from usefulness, will also be explored. Information will be collected primarily through semi-structured interviews with participants (collaborators and stakeholders). Existing scales measuring environmental attitudes and preferences for science in decision-making and general attitudes toward science will be used so comparisons with larger national and international samples can be made. In addition, scoping and development meetings will be observed to determine how shared understanding of user needs is developed and then framed as a visualization problem.

A longstanding problem in the design of cyber-physical systems is the inability and ineffectiveness in coping with software and hardware evolutions over the lifetime of a design or across multiple versions in the same product family. The objective of this project is to develop a systematic framework for designing extensible cyber-physical systems that can enable efficient and correct updates with minimal redesign and re-verification efforts. The intellectual merits are (1) a new and unified framework that optimizes system extensibility by addressing both functional correctness and platform feasibility, and (2) new algorithms for functional verification with platform consideration, software architecture synthesis driven by extensibility metrics, and integration of verification and synthesis for joint design space exploration. The project?s broader significance and importance are (1) enabling engineers to cope with continual changes in cyber-physical design components or operating conditions, thereby significantly reducing redesign and re-verification cost, (2) providing a general framework for designing extensible systems that is applicable to a wide range of systems including robotic, automotive, and avionic systems, and (3) providing new methodologies and techniques that facilitate the training of undergraduate and graduate students to meet the design challenges of cyber-physical systems. Many cyber-physical systems today are one-off designs -- systems designed without future changes in mind. The proposed extensibility-driven design (EDD) framework treats extensibility as a first-class design objective and addresses it with a holistic consideration of functional properties and platform implementation. An EDD design flow provides the following capabilities. At the initial design stage, EDD identifies certain constraints (e.g., timing) that are critical for functional correctness, and explores the design space to maximize the amount of future software and hardware changes that can be made without violating these constraints. During design updates, EDD first determines whether it is possible to accommodate the updates through software architecture re-synthesis, so as to avoid costly re-verification. In the cases where the updates violate existing platform constraints and requirements, EDD selectively modifies some of them to explore feasible changes while minimizing re-verification efforts.

A longstanding problem in the design of cyber-physical systems is the inability and ineffectiveness in coping with software and hardware evolutions over the lifetime of a design or across multiple versions in the same product family. The objective of this project is to develop a systematic framework for designing extensible cyber-physical systems that can enable efficient and correct updates with minimal redesign and re-verification efforts. The intellectual merits are (1) a new and unified framework that optimizes system extensibility by addressing both functional correctness and platform feasibility, and (2) new algorithms for functional verification with platform consideration, software architecture synthesis driven by extensibility metrics, and integration of verification and synthesis for joint design space exploration. The project?s broader significance and importance are (1) enabling engineers to cope with continual changes in cyber-physical design components or operating conditions, thereby significantly reducing redesign and re-verification cost, (2) providing a general framework for designing extensible systems that is applicable to a wide range of systems including robotic, automotive, and avionic systems, and (3) providing new methodologies and techniques that facilitate the training of undergraduate and graduate students to meet the design challenges of cyber-physical systems. Many cyber-physical systems today are one-off designs -- systems designed without future changes in mind. The proposed extensibility-driven design (EDD) framework treats extensibility as a first-class design objective and addresses it with a holistic consideration of functional properties and platform implementation. An EDD design flow provides the following capabilities. At the initial design stage, EDD identifies certain constraints (e.g., timing) that are critical for functional correctness, and explores the design space to maximize the amount of future software and hardware changes that can be made without violating these constraints. During design updates, EDD first determines whether it is possible to accommodate the updates through software architecture re-synthesis, so as to avoid costly re-verification. In the cases where the updates violate existing platform constraints and requirements, EDD selectively modifies some of them to explore feasible changes while minimizing re-verification efforts.

Many areas of the United States are subject to seasonal and cyclical natural disasters like floods, earthquakes and hurricanes, while all areas may experience technological or human-caused events leading to communications disruptions. Following a disaster, it is essential for professional emergency responders to have a comprehensive understanding of the damage in the community in order to prioritize resources to save lives and protect the environment. Failure to develop an accurate picture of community conditions may lead to ineffective allocation of scarce response and rescue resources. Current technologies used for day-to-day emergency response information gathering from the public, such as 9-1-1 calls and social media, are often disrupted by the disaster?s impact, which may persist for days after an event. One of the key factors enabling a coordinated emergency response and community resilience to disaster is rapid communication from community members such as residents, businesses, schools and hospitals to public safety services about community conditions, such as the location of trapped people, collapsed buildings, fires and hazardous materials accidents, highway damage, and traffic congestion. Robust and resilient communication systems incorporating and enhancing existing technologies are the solution. The City of San Jose has recognized the likelihood of post-disaster information deficits which can be resolved through increased connectivity of diverse community elements to public safety communications. Recognizing the presence of privately-owned Smart phones throughout the community, the City is seeking an information gathering and dissemination solution that would enable Smart phone users to maintain communication with public safety services even in disaster conditions. San Jose State University, partnered with the San Jose Office of Emergency Services (OES), proposes to develop a novel method for maintaining connectivity for residents to public safety services. The proposed connectivity and networking technologies will keep citizens connected to vital services and information, and allow them to provide disaster assessment information to public safety agencies. This project will also create a cloud dashboard for emergency responders, and create a comprehensive view of community conditions which leads to an effective emergency response. The prototype system will enable the city's public safety agencies to prioritize emergency response demands and respond quickly, and minimize the catastrophic impact on the City of San Jose and its community and economy. The prevalence of disruptive events across the United States makes the development of a resilient communication solution imperative. The available collaboration with the City of San Jose provides a real-world partner and testbed for new technology applications with nation-wide application potential. As climates change, storms become stronger, sea levels rise, the electricity grid ages and social disruptions increase, time is of the essence for creating a resilient and accessible solution to reliable communication connectivity. This Early Concept Grant for Exploratory Research (EAGER) will solve the key challenges that must be tackled to achieve this timeliness and provide strategies and system solutions to spur emergency awareness, management, and preparedness. Finally, all code and data in this project will be released openly, supporting future research, development, and training. First responders to disasters need a complete picture of the community's status in order to accurately assess the condition of the inhabitants and organize available resources to save lives, protect the environment and prevent further damage in the community. In normal circumstances public safety services rely on 9-1-1 calls and social media to gather information from residents about community conditions. However, under disaster conditions, these normal communication methods will be interrupted, including landline and cell phones, internet connectivity and power. In these circumstances, novel systems must be available to substitute for the lost connectivity, to allow residents to connect to the public safety answering point, and to allow the Emergency Operations Center to collect and aggregate critical information across sectors to ensure that lifesaving operations are conducted expeditiously. The solution to managing risks to disaster-prone communities includes integrating existing technologies, applications, data and e-services in sustainable networks that will support emergency communications even in catastrophic events. This research proposes to develop a community infrastructure for interoperable emergency connectivity that can operate in austere conditions, provide its own power, and create linkages throughout the community and across jurisdictional boundaries. This project will deploy the edge devices in local communities with multi-modal communication modules as well as an external long range radio. The proposed resilient and participatory networking framework on top of the remote edge devices will enable collaborative communication as well as participatory sensing. To solve current deficiencies in the ability of allowing city emergency responders to control and automate the remote edge devices, this project extends existing cloud orchestration frameworks to edge devices that are agnostic to the network media. For this demonstration project, the central cloud deployed in the City of San Jose?s Emergency Operations Center will control the remote edge devices, and be responsible for resilient quality testing, automatic validation, disaster assessment, resource allocation, and the automation of remote edge devices.

The goal of this project is to demonstrate that advanced information and sensor technology can improve operational efficiency and increase the security and safety of fire fighters. Existing firefighting systems will be augmented by exploiting the information capabilities of hardware and software components that can be attached to the existing fire fighter equipment, with minimal physical burden and required training. The system will provide a model of the emergency scenario that will allow the commander to evaluate possible alternative actions based on their experience and available resources. This situation awareness will be created from the data provided by the fighter gear (microphones, cameras, body and ambient sensors), and will include the estimation of the fighter situation (including fighters' incidents, oxygen reserve or estimated time left to leave the scenario) and the scenario itself (including the presence of victims, evaluation hazardous object or environments, as hot surfaces, toxic gas and others). This proposal is highly relevant for smart and connected communities. It addresses problem space of great relevance in emergency operations with a technology solution that faces significant research and operational challenges. The project engages technical communities, non-profit partners and local government institutions. It is a cooperation between various departments of the University of New Mexico, in collaboration with the City of Santa Fe and the City of Albuquerque Fire departments and the National Fire Protection Association. The project integrates a hardware layout that collects the data from each fire fighter on duty with a software engine for extracting data and processing. Data from infrared cameras, body and ambient sensors' will be interfaced to a communication node to transmit extracted and compressed information using a mesh structure for communications based on software defined radio supporting heterogeneous communication assets, instant deployment and hot reconfiguration, resiliency and recovery abilities. The software engine will integrate machine learning based feature extraction and prediction methods that will process the ambient data, audio and speech, to sense the fighter's condition, detect relevant keywords whose meaning can be transmitted, or give orders to the system. Video will be locally processed to extract relevant features (civilians, heat surfaces, hazardous objects and others). Machine learning algorithms will then be used to construct the situational awareness that will be served to the commander and fire fighters, including scenario and fire fighters' situation. The system will be tested in a variety of operational scenarios to evaluate the potential for transition and application in other emergency domains.

Software-Defined Control (SDC) is a revolutionary methodology for controlling manufacturing systems that uses a global view of the entire manufacturing system, including all of the physical components (machines, robots, and parts to be processed) as well as the cyber components (logic controllers, RFID readers, and networks). As manufacturing systems become more complex and more connected, they become more susceptible to small faults that could cascade into major failures or even cyber-attacks that enter the plant, such as, through the internet. In this project, models of both the cyber and physical components will be used to predict the expected behavior of the manufacturing system. Since the components of the manufacturing system are tightly coupled in both time and space, such a temporal-physical coupling, together with high-fidelity models of the system, allows any fault or attack that changes the behavior of the system to be detected and classified. Once detected and identified, the system will compute new routes for the physical parts through the plant, thus avoiding the affected locations. These new routes will be directly downloaded to the low-level controllers that communicate with the machines and robots, and will keep production operating (albeit at a reduced level), even in the face of an otherwise catastrophic fault. These algorithms will be inspired by the successful approach of Software-Defined Networking. Anomaly detection methods will be developed that can ascertain the difference between the expected (modeled) behavior of the system and the observed behavior (from sensors). Anomalies will be detected both at short time-scales, using high-fidelity models, and longer time-scales, using machine learning and statistical-based methods. The detection and classification of anomalies, whether they be random faults or cyber-attacks, will represent a significant contribution, and enable the re-programming of the control systems (through re-routing the parts) to continue production. The manufacturing industry represents a significant fraction of the US GDP, and each manufacturing plant represents a large capital investment. The ability to keep these plants running in the face of inevitable faults and even malicious attacks can improve productivity -- keeping costs low for both manufacturers and consumers. Importantly, these same algorithms can be used to redefine the production routes (and machine programs) when a new part is introduced, or the desired production volume is changed, to maximize profitability for the manufacturing operation .