Cyber-physical systems of the near future will collaborate with humans. Such cognitive systems will need to understand what the humans are doing. They will need to interpret human action in real-time and predict the humans' immediate intention in complex, noisy and cluttered environments. This proposal puts forward a new architecture for cognitive cyber-physical systems that can understand complex human activities, and focuses specifically on manipulation activities. The proposed architecture, motivated by biological perception and control, consists of three layers. At the bottom layer are vision processes that detect, recognize and track humans, their body parts, objects, tools, and object geometry. The middle layer contains symbolic models of the human activity, and it assembles through a grammatical description the recognized signal components of the previous layer into a representation of the ongoing activity. Finally, at the top layer is the cognitive control, which decides which parts of the scene will be processed next and which algorithms will be applied where. It modulates the vision processes by fetching additional knowledge when needed, and directs the attention by controlling the active vision system to direct its sensors to specific places. Thus, the bottom layer is the perception, the middle layer is the cognition, and the top layer is the control. All layers have access to a knowledge base, built in offline processes, which contains the semantics about the actions. The feasibility of the approach will be demonstrated through the development of a smart manufacturing system, called MONA LISA, which assists humans in assembly tasks. This system will monitor humans as they perform assembly task. It will recognize the assembly action and determine whether it is correct and will communicate to the human possible errors and suggest ways to proceed. The system will have advanced visual sensing and perception; action understanding grounded in robotics and human studies; semantic and procedural-like memory and reasoning, and a control module linking high-level reasoning and low-level perception for real time, reactive and proactive engagement with the human assembler. The proposed work will bring new tools and methodology to the areas of sensor networks and robotics and is applicable, besides smart manufacturing, to a large variety of sectors and applications. Being able to analyze human behavior using vision sensors will have impact on many sectors, ranging from healthcare and advanced driver assistance to human robot collaboration. The project will also catalyze K-12 outreach, new courseware (undergraduate and graduate), publication and open-source software.

This proposal addresses the safety and security issues that arise when giving users remote-access to a multi-robot research test-bed, where mobile robots can coordinate their behaviors in a collaborative manner. Through a public interface, users are able to schedule, and subsequently upload, their own code and run their experiments, while being provided with the scientific data produced through the experiment. Such an open-access framework has the potential to significantly lowering the barriers to entry in robotics research and education, yet is inherently vulnerable from a safety and security point-of-view. This proposal aims at the development and definition of appropriate cyber-physical security notions, formal verification algorithms, and safety-critical, real-time control code for teams of mobile robots that will ultimately make such a system both useful and safe. On top of the research developments, this proposal contains a Transition to Practice component that will allow the system to become a highly usable, shared test-bed; one that can serve as a model for other open, remote-access test-beds. Safety is of central importance to the successful realization of any remote-access test-bed and failure to enforce safety could result in injury in local operators and damaged equipment. To guarantee safe operation, while allowing users to test algorithms remotely, new science is required in the domain of safety-critical control. To address this need, the proposed work follows a three-pronged approach, namely (1) development and use of novel types of barrier certificates in the context of minimally invasive, optimization-based controllers with provable safety properties, (2) formal methods for verification of safety-critical control code for networked cyber-physical systems, and (3) novel methods for protecting against machine-to-machine cyber attacks. By bringing together ideas from multi-agent robotics, safety-critical control, formal verification, and cyber-security, this project will result in a unified and coherent approach to security in networked cyber-physical systems. The potential impact of the resulting open-access multi-robot test-bed is significant along the research, education, and general outreach dimensions in that a future generation of roboticists at institutions across the country will have open and remote access to a world-class research facility, and educators at all levels will be able to run experiments on actual robots.

This proposal addresses a multidisciplinary workshop with academic researchers, corporate technology providers, and agricultural producers to define research challenges and a research road-map to address the following major FEWS challenges: 1. Developing novel targeted remote sensing and in-situ sensing technology that can be practically fielded and used in food and water system management. 2. Developing novel integrated hydrology, soil, microclimate, and plant/agricultural production models that interact accurately and across traditional scales for understanding local, regional, and national impacts. 3. Turning this developing and pending FEWS data deluge into usable, actionable information for agricultural producers, local and regional decision makers, and citizens. The workshop addresses the emerging issues in the food/water/energy system throughout the diverse geography of United States and over various crops and environmental conditions to better understand and model, and ultimate devise a solution for the changes to the FEWS system. The solution must be multifaceted, multidisciplinary in order to incorporate sensing, hydrology, visual analytics, and the potential for increased climate change. The workshop will generate a report and other artifacts that will lead to research into solving these challenges and have an impact on scientific fields including, sensing technology, hydrology, soil science, climate, data fusion, analysis, visualization, and data driven decision making, as well as agricultural production, local and regional economies, sustainability and planning.

Inadequate system understanding and inadequate situational awareness have caused large-scale power outages in the past. With the increased reliance on variable energy supply sources, system understanding and situational awareness of a complex energy system become more challenging. This project leverages the power of big data analytics to directly improve system understanding and situational awareness. The research provides the methodology for detecting anomalous events in real-time, and therefore allow control centers to take appropriate control actions before minor events develop into major blackouts. The significance for the society and for the power industry is profound. Energy providers will be able to prevent large-scale power outages and reduce revenue losses, and customers will benefit from reliable energy delivery with service guarantees. Students, including women and underrepresented groups, will be trained for the future workforce in this area. The project includes four major thrusts: 1) real-time anomaly detection from measurement data; 2) real-time event diagnosis and interpretation of changes in the state of the network; 3) real-time optimal control of the power grid; 4) scientific foundations underpinning cyber-physical systems. The major outcome of this project is practical solutions to event or fault detection and diagnosis in the power grid, as well as prediction and prevention of large-scale power outages.

Laboratory-on-a-chip (LoC) technology is poised to improve global health through development of low-cost, automated point-of-care testing devices. In countries with few healthcare resources, clinics often have drugs to treat an illness, but lack diagnostic tools to identify patients who need them. To enable low-cost diagnostics with minimal laboratory support, this project will investigate domain-specific LoC programming language and compiler design in conjunction with device fabrication technologies (process flows, sensor integration, etc.). The project will culminate by building a working LoC that controls fluid motion through electronic signals supplied by a host PC; a forensic toxicology immunoassay will be programmed in software and executed on the device. This experiment will demonstrate benefits of programmable LoC technology including miniaturization (reduced reagent consumption), automation (reduced costs and uncertainties associated with human interaction), and general-purpose software-programmability (the device can execute a wide variety of biochemical reactions, all specified in software). Information necessary to reproduce the device, along with all software artifacts developed through this research effort, will be publicly disseminated. This will promote widespread usage of software-programmable LoC technology among researchers in the biological sciences, along with public and industrial sectors including healthcare and public health, biotechnology, water supply management, environmental toxicity monitoring, and many others. This project designs and implements a software-programmable cyber-physical laboratory-on-a-chip (LoC) that can execute a wide variety of biological protocols. By integrating sensors during fabrication, the LoC obtains the capability to send feedback in real-time to the PC controller, which can then make intelligent decisions regarding which biological operations to execute next. To bring this innovative and transformative platform to fruition, the project tackles several formidable research challenges: (1) cyber-physical LoC programming models and compiler design; (2) LoC fabrication, including process flows and cyber-physical sensor integration; and (3) LoC applications that rely on cyber-physical sensory feedback and real-time decision-making. By constructing a working prototype LoC, and programming a representative feedback-driven forensic toxicology immunoassay, the project demonstrates that the proposed system can automatically execute biochemical reactions that require a closed feedback loop. Expected broader impacts of the proposed work include reduced cost and increased reliability of clinical diagnostics, engagement with U.S. companies that use LoC technology, training of graduate and undergraduate students, increased engagement and retention efforts targeting women and underrepresented minorities, student-facilitated peer-instruction at UC Riverside, a summer residential program for underrepresented minority high-school students at the University of Tennessee, collaborations with researchers at the Oak Ridge National Laboratory, and creation, presentation, and dissemination of tutorial materials to promote the adoption and use of software-programmable LoCs among the wider scientific community.

This award provides NSF support to sponsor students and faculty mentors from US institutions to participate in a doctoral symposium held during the First Workshop on Privacy and Security in Pervasive e-Health and Assistive Environments in Corfu, Greece, June 9-13, 2009. This workshop is in conjunction with the Second International Conference on Pervasive Technologies Related to Assistive Environments (PETRA?09 http://www.petrae.org). The doctoral symposium shares the aims of the PSPAE workshop, seeking new directions in security and privacy research and education. The focus of the PSPAE workshop is a multidisciplinary discussion aimed towards developing new insights into privacy and security issues for pervasive technologies that will be deployed for the benefit of society. The scientific community is expected to benefit from discussions on the crossover between security and privacy and on the issues arising in complex application areas such as healthcare. The doctoral consortium seeks to open new horizons for graduate students who may pursue privacy and security research in healthcare and other pervasive assistive environments. Another objective is to create an international forum that can expose graduate students to the differences in privacy policies around the world, especially regarding health privacy and security.

Cyber-physical systems (CPS) are characterized by extremely tight integration of and coordination between computational and physical resources. CPS integrate computation, communication, and storage capabilities through systems of systems that must interact with the physical world in real-time at multiple time scales and often at multiple spatial scales. The inherent heterogeneity and the non-deterministic operation of different components in these systems pose new challenges to traditional control, communication, real-time scheduling, and robotics disciplines. In conjunction with the IEEE Real-Time Systems Symposium in 2009 (RTSS 2009), this project helps to support a Ph.D. student forum to discuss (i) the set of interdisciplinary research problems that arise in the context of cyber-physical systems, (ii) novel applications that become possible thanks to the integration of computing, communication, and interaction with the physical world at scale, and (iii) initial system architecture that addresses some of these research problems. The primary goal is to help students (and the real-time community) recognize that cyberphysical systems are different from the over-engineered real-time embedded systems of the past, and to provide a forum by which students can discuss their proposal for addressing the complicated aggregate systems issues that arise in this context. As such, we need to encourage constructive debate on emerging research topics. A secondary goal is to encourage student involvement in new research directions and offer a channel to discuss and reward the most innovative student ideas in this exciting emerging research field. Advisors and students will be welcome to attend the forum, but the focus will be on training and motivating the next generation of researchers.