This project will develop novel, body-worn, flexible sensors fabricated using low-cost inkjet printing technology on thin film polymers, develop novel algorithms capable of automatically detecting health events in different contexts, and develop a novel data reliability metric by analyzing sensor and context data in real-time. The project will produce practice components for test and validation in a clinical setting with cardiac patients to determine their effectiveness for monitoring heart conditions. If successful, the project will provide patients and clinicians with a tool to improve health monitoring in natural environments. The research is expected to impact additive manufacturing methods, flexible electronics, health monitoring, and smart and connected communities initiatives. It will also provide training for undergraduate and graduate students and expose the next generation of scholars and workers to these technologies through a Summer Code Camp for high school students.

Next generation Cyber-Physical Systems (CPS) must utilize resilient and reliable cyber/physical interfacing, be economically viable, and be capable of processing extremely large data automatically and reliably. Achieving this requires overcoming current technological barriers associated with seamless integration of computation and physical domains and meaningful interpretation of multimodal and multigrain data of scalable CPS. Balancing theory with experimentation, this project will: 1) produce foundational engineering process for CPS interface with thin-film flexible electronic electrodes and low cost sensors fabricated with inkjet printing; 2) develop new algorithms for autonomous processing of sensor data to detect context-aware events of interest and data reliability metric for closed-loop CPS using real-time machine learning implemented at edge; and 3) deploy CPS practice components in a real-life pilot study to explore detection of cardiac episodes and explore various closed loop feedback approaches.

With the growing world population and diminishing agricultural lands, it becomes imperative to maximize crop yield by protecting crop health and mitigating against pests and diseases. Though there are decades-old practices still in place, there is also growing adoption of so-called precision agriculture solutions, which employ emerging technologies in sensing, automation, and analytics in daily farmland operations. As farmers gain real-time access to critical data (e.g., land and weather conditions) and can quickly share any untoward findings with others, farmland operations are morphing into full-fledged cyber-physical systems. To this end, this project seeks to develop, implement and evaluate a multi-robot agricultural information collection system that is autonomous, efficient and secure.

This project led by the University of North Florida (UNF) and supported by the University of Central Florida (UCF) has two main goals: (i) develop and implement novel information collection techniques for autonomous mobile robots that collect, store and share data in an efficient yet secure manner using blockchain,and (ii)and to train undergraduate and graduate students to conduct basic and applied research while closely working with local farmland partners in north-east Florida. Current technologies already use robots for agricultural purposes, but they typically have a high maintenance cost and do not necessarily consider issues related to security and data integrity. The primary objective is to design and deploy a set of autonomous robots that communicate wirelessly and navigate through planned paths in order to collect valuable data. This project will also consider the threat of security attacks by which collected data can be corrupted; seeking new distributed blockchain-based consensus protocols that mitigate the adversarial influence of such attacks. This project also contains a significant research and education component leveraging the leadership of UNF in the context of a primarily undergraduate institution (RUI). Being predominantly an undergraduate institution, there is a lack of opportunity for pursuing higher degrees in the Jacksonville area. This project aligns with an established Memorandum of Understanding (MoU) between UNF and UCF to provide a conduit for computing/engineering students to pursue M.S. degrees at UNF that feed seamlessly into Ph.D. programs at UCF. Students will benefit from the new robotics course to be developed at UNF and the ones being offered at UCF. Research progress will be showcased via technical workshops at both institutions to be held annually. Developed solutions are expected to transfer to other cyber-physical system applications, including search and rescue, patrolling, advanced manufacturing, among others. Most broadly, this project will raise awareness among today's teenagers and young adults of the impending agricultural crisis if worldwide food production falls even further behind meeting demands of an increasing global population.

The goal of this research is to enable a broad spectrum of programmers to successfully create apps for distributed computing systems including smart and connected communities, or for systems that require tight coordination or synchronization of time. Creating an application for, say, a smart intersection necessitates gathering information from multiple sources, e.g., cameras, traffic sensors, and passing vehicles; performing distributed computation; and then triggering some action, such as a warning. This requires synchronization and coordination amongst multiple interacting devices including systems that are Internet of Things (IoT) devices that may be connected to safety critical infrastructure. Rather than burden the programmer with understanding and dealing with this complexity, we seek a new programming language, sensor and actuator architecture, and communications networks that can take the programmer's statements of "what to do" and "when to do", and translate these into "how to do" by managing mechanisms for synchronization, power, and communication. This approach will enable more rapid development of these types of systems and can have significant economic development impact.

The proposed approach has four parts: (1) creating a new programming language that embeds the notion of timing islands -- groups of devices that cooperate and are occasionally synchronized; (2) creating a network-wide runtime system that distributes and coordinates the action of code blocks -- portions of the program -- across devices; (3) extending the capabilities of communications networks to improve the ability to synchronize devices and report the quality of synchronization back to the runtime system, enabling adaptive program behavior; and (4) extending device hardware architecture to support synchronization and time-respecting operation.

This NSF CPS CAREER project studies the hardware/software co-design of sub-millisecond machine learning control for high-rate dynamic systems with non-stationary inputs that change the system?s state (i.e., damage). Such systems include combustion processes in jet engines, vehicle structures during crashes, and active blast mitigation structures. The novelty of the approach taken in this project is to co-design the control systems with the computing hardware they will run on to constrain system latency to within 1 millisecond. The developed solutions will be able to learn a system?s dynamics at the data rates required by high-rate dynamic systems. Machine learning models will learn the dynamics of the non-linear system online, which will then be used to model the dynamics of the system to the appropriate prediction horizon. The project is developing an automated programming methodology that enables the deployment of these real-time controllers onto compact and power-efficient computing devices. It follows that this research will impact society and the mission of the NSF by enabling a better understanding of dynamic systems operating in high-rate environments while enabling intelligent decision-making capabilities at speeds never before reached. The project will leverage existing and valuable resources at the University of South Carolina to involve several high school and undergraduate students in the project; with emphasis on providing research experiences to underrepresented, first-generation, and low-income students. This project will also train Ph.D. students in real-time machine learning and control.

More specifically, this research is addressing the fundamental question of how programmable hardware can be used to enable machine learning and control for systems that demand ultra-low latency. This is being done by formulating a framework for real-time machine learning control that co-designs hardware and software and provides a path to deployment on field programmable gate arrays (FPGAs). The project is: 1) Training a novel long short-term memory (LSTM) model on-chip with a custom online trainer that maps sensor signal and actuator input for a high-rate system to system state in real-time. 2) Developing approaches to share FPGA signal processing and memory resource for the parallel utilization of multiple LSTM forward-pass cores while maintaining deterministic timing. 3) Studying trade-offs between accuracy, performance, and resource requirements for real-time machine learning control at the microsecond timescale. Validation of the developed approach is being performed using a hardware-in-the-loop testing methodology with fast-acting actuators to control the outer mold line of a structural panel in simulated hypersonic flight.

The interactions of light with objects in a scene are often complex. An image --- which only captures 2D spatial variations --- is poorly equipped to unravel these interactions and infer properties of a scene including its shape, reflectance, and its composition. This is especially true for scenes that have sharp reflections, refractions, and volumetric scattering. This research models interactions of light with scenes using light rays and their transformations. The central hypothesis underlying the research is the idea that problems of shape, reflectance and material composition estimation are often simpler and well-posed when they are studied using light rays and their transformations. A wide-range of real-world objects and scenes stand to benefit from progress made in this research; this includes scenes with complex configurations that lead to inter-reflections, objects with shine, specularities, and spatially-varying reflectances, as well as objects that are transparent, or translucent. A diverse set of applications including machine vision, microscopy, and consumer photography stand to benefit from this research. The education and outreach components of this project disseminates image processing research in the broader Pittsburgh area via camera building workshops and lab demos for middle/high-school students, and professional development courses for physics teachers.

The focus of the research is to develop novel acquisition and processing methods for scene understanding by studying characterizations of light that go beyond images. In particular, the research analyzes the properties of two signals: the plenoptic function, which captures spatial, temporal, angular, and spectral variations of light, and the plenoptic light transport, which captures how light propagates through a scene. The central hypothesis of the research is that the plenoptic function and light transport provide a rich encoding of how light interacts with a scene; hence, unlike image-based inference, plenoptic inference can be fundamentally well-conditioned even for scenes that interact with light in a complex manner. To this end, the research develops novel low-dimensional models for plenoptic functions that are based on physical laws governing interaction of light with a scene. The research also builds novel computational cameras that acquire light propagates in a scene by decomposing into light paths of varying complexity, and subsequently estimating the 3D shape, reflectance, and material composition.

Wearable sensors show much promise for medical, sports, defense, emergency, and consumer applications, but are currently limited to obtrusive implementations. Akin to the evolution of cell phones that evolved from foot-long prototypes to recent smart devices, next-generation wearables are envisioned to be seamlessly embedded in fabrics. This CAREER project aims to understand the unique challenges of operating such textile sensors ?in-the-wild? and to empower their reliable operation via closed-loop interaction among fabrics, electronics, and humans. To serve as a model and to inspire new applications, the project focuses on new classes of functionalized garments that can seamlessly monitor kinematics and/or tissue abnormalities with unique advantages over the state-of-the-art. Concurrently, the integrated education/outreach efforts aim to increase student and public exposure to bio-electromagnetics that are now confined to specialized research, yet can enable interdisciplinary training for all via appealing activities with direct societal impact.

This CAREER project will pioneer a design, modeling, and implementation framework that reconciles human-in-the-loop Cyber-Physical Systems (CPS) with conductive e-textile sensors operating in complex (human wearing a sensing fabric) and dynamic (real-world) environments. Cognitive and fully-adaptive e-textile CPS are proposed that: (a) are cognizant of inputs received by the wearer, the fabric, and the environment, and (b) integrate agility in both the cyber and physical sides for closed-loop adaptability on the fly. In turn, potentials in optimizing performance, minimizing resources, and enhancing opportunities for myriads of human-in-the-loop CPS are envisioned to be significant. Without loss of generality, focus is on a novel-multi-utility sensor that addresses two of the most challenging sensing modalities in the area of wearables, i.e., motion capture and tissue abnormality monitoring. These modalities may be individually or concurrently employed to create models for dense-data (motion captured on the go), sparse-data (tissues monitored over sparse intervals), and context-aware (both of the above) human-in-the-loop CPS. As a case study, a novel CPS will be progressively designed ? from concept to in vivo testing ? to improve outcomes after anterior cruciate ligament reconstruction.

Cognitive workload refers to the level of mental effort put forth by an individual in response to a cognitive task. Unfortunately, no technology currently exists that can monitor an individual?s levels of cognitive workload in real-world environments using a seamless, reliable, and low-cost approach. We propose to fill this gap by using a novel magnetocardiography (MCG) system worn upon the subject?s chest to allow the sensor to collect the magnetic fields that are naturally emanated by the heart and associated with brain activity. This science is anticipated to greatly accelerate progress in such diverse disciplines as pediatric concussion recovery, pilot training, improved user-machine interfaces, injury prevention in construction environments, increased human performance in risky missions, and improved education outcomes. In addition to advances in basic science, the proposed research is expected to be of significant interest to students and the public. Through targeting interdisciplinary education and diverse recruitment, we intend to expose new audiences to STEM concepts via workshops and family-friendly outings. 

The proposed MCG sensor is smartly integrated in a Cyber-Physical System (CPS) with two inter-connected loops: (a) a human-in-the-loop that addresses changes in the thresholds of different cognitive states as a function of time, and (b) a non-human-in-the-loop that adapts the system?s algorithmic and hardware components for high-accuracy classification of cognitive workload with minimum resource usage. Our goals are to: (1) Build a knowledgebase concerning the impact of hardware/algorithmic advances upon MCG sensor performance in real-world settings. (2) Explore the classification of cognitive workload from MCG data and close the loop with the wearer for dynamic calibrations that address the time-varying thresholds of cognitive states. (3) Ensure operability in dynamic real-world settings and close the loop between the cyber and physical sides for minimal resource usage. (4) Validate the CPS within the framework of measuring cognitive workload for children with concussion. Without loss of generality, we select this population given the immense clinical potential: the effects of cognitive activity on pediatric concussion recovery are currently unknown, largely due to the difficulties in quantifying cognitive activity workload.

Increases in temperatures and drought duration and intensity due to climate change, together with the expansion of wildlife-urban interfaces, has dramatically increased the frequency and intensity of forest fires, and has had devastating effects on lives, property, and the environment. To address this challenge, this project?s goal is to design a network of airborne drones and wireless sensors that can aid in initial wildfire localization and mapping, near-term prediction of fire progression, and providing communications support for firefighting personnel on the ground. Two key aspects differentiate the system from prior work: (1) It leverages and subsequently updates detailed three-dimensional models of the environment, including the effects of fuel type and moisture state, terrain, and atmospheric/wind conditions, in order to provide the most timely and accurate predictions of fire behavior possible, and (2) It adapts to hazardous and rapidly changing conditions, optimally balancing the need for wide-area coverage and maintaining communication links with personnel in remote locations. The science and engineering developed under this project can be adapted to many applications beyond wildfires including structural fires in urban and suburban settings, natural or man-made emergencies involving radiation or airborne chemical leaks, "dirty bombs" that release chemical or biological agents, or tracking highly localized atmospheric conditions surrounding imminent or on-going extreme weather events.

The system developed under this project will enable more rapid localization and situational awareness of wildfires at their earliest stages, better predictions of both local, near-term and event-scale behavior, better situational awareness and coordination of personnel and resources, and increased safety for fire fighters on the ground. Models ranging from simple algebraic relationships based on wind velocity to more complex time-dependent coupled fluid dynamics-fire physics models will be used to anticipate fire behavior. These models are hampered by stochastic processes such as the lofting of burning embers to ignite new fires, that cause errors to grow rapidly with time. This project is focused on closing the loop using sensor data provided by airborne drones and ground-based sensors (GBS). The models inform the sensing by anticipating rapid growth of problematic phenomena, and the subsequent sensing updates the models, providing local wind and spot fire locations. Closing this loop as quickly as possible is critical to mitigating the fire?s impact. The system we propose integrates advanced fire modeling tools with mobile drones, wireless GBS, and high-level human interaction for both the initial attack of a wildfire event and subsequent on-going support.

Today, operators of cellular networks and electricity grids measure large volumes of data, which can provide rich insights into city-wide mobility and congestion patterns. Sharing such real-time societal trends with independent, external entities, such as a taxi fleet operator, can enhance city-scale resource allocation and control tasks, such as electric taxi routing and battery storage optimization. However, the owner of a rich time series and an external control authority must communicate across a data boundary, which limits the scope and volume of data they can share. This project will develop novel algorithms and systems to jointly compress, anonymize, and price rich time series data in a way that only shares minimal, task-relevant data across organizational boundaries. By emphasizing communication efficiency, the developed algorithms will incentivize data sharing and collaboration in future smart cities.

The key motivation of this work is that today's representations of time series data are designed independently of an ultimate control task, which often causes unnecessary temporal features to be sent, private features to be revealed, and the most salient trends to be under-valued. Accordingly, this project will develop a unified approach to co-design succinct, private representations of rich time series data along with an ultimate control task. Here, co-design means that the forecast representation is learned within the broader context of a control objective while accounting for bandwidth constraints, privacy, and economic costs and incentives for data processing. The algorithms will compute a controller's sensitivity to prediction errors, which can arise from data compression, forecast uncertainty, as well as artificial noise injected by modern privacy tools. Crucially, the controller's sensitivity will in turn be relayed to a network operator to guide its optimization and learning (e.g., co-design) of a concise, task-relevant forecast representation that masks private attributes and naturally prices temporal features by their importance to control. The research will, for example, enable operators to flexibly use the same underlying cell demand data to emphasize peak-hour variability for taxi routing, while seamlessly delivering fine-grained throughput forecasts to a mobile video streaming company without revealing private user mobility. Finally, the case studies in this project will be integrated into courses on learning-based control at UT Austin and Cornell. Broader impacts also include outreach and inclusion efforts to engage students from groups that have historically been under-represented in STEM fields.
 

The proposed decentralized/distributed control and optimization for the critical cyber-physical networked infrastructures (CPNI) will improve the robustness, security and resiliency of the electric distribution grid, which directly impacts the life of citizens and national economy. The proposed control and optimization architectures are flexible, adapt to changing operating scenarios, respond quickly and accurately, provide better scalability and robustness, and safely operate the system even when pushed towards the edges by leveraging massive sensor data, distributed computation, and edge computing. The algorithms and platform will be released open source and royalty-free and the project team will work with industry members and researchers for wider usage of the developed algorithms for other CPNI. Developed artifacts as part of the proposed work will be integrated in existing undergraduate and graduate related courses. Undergraduate students will be engaged in research through supplements and underrepresented and pre-engineering students will be engaged through existing outreach activities at home institutions including Imagine U program and 4-H Teens summer camp programs and the Pacific Northwest Louis Stokes Alliance for Minority Participations. Additionally, project team plans to organize a workshop in the third year to demonstrate the fundamental concepts and applications of the proposed control and optimization architecture to advance CPNI. Developed solutions can be extended for range of applications in multiple CPNIs beyond use cases discussed in the proposed work.

While the proposed control architecture with edge computing offer great potential; coordinating decentralized control and optimization is extremely challenging due to variable network and computational delays, several interleavings of message arrivals, disparate failure modes of components, and cyber security threats leading to several fundamental theoretical problems. Proposed work offers number of novel solutions including (a) adaptive and delay-aware control algorithms, (b) Predictive control and distributed optimization with realistic cyber-physical constraints, (c) threat sharing, data-driven detection and mitigation for cyber security, (d) coordination and management of computing nodes, (e) knowledge learning and sharing. Proposed solutions will be a step towards advancing fundamentals in CPNI and in engineering next generation CPNI. The proposed work also aims to use high fidelity testbed to evaluate developed algorithms and tools for specific CPNI: electric distribution grid.