Perception systems are a key component of modern autonomous cyber-physical systems (CPS), from self-driving vehicles to autonomous robots and drones. For instance, for a self-driving vehicle, perception systems provide functionalities such as estimating the state of the vehicle, building a map of obstacles in its surroundings, and detecting and tracking external objects and pedestrians. As exemplified by recent self-driving car accidents, perception failures can cascade to catastrophic system failures and compromise human safety. Therefore, the development of trustworthy perception systems is paramount to ensure safety and enable adoption of high-integrity and safety-critical CPS applications. This project lays the foundations of certifiable perception by developing a toolkit of theory, algorithms, and implementations to monitor and drastically reduce subsystem and system-level failures of perception. In particular, this project will (i) develop a new class of certifiable perception algorithms that operate reliably in challenging conditions, are equipped with input-output contracts describing their functionalities, and can formally assert contract satisfaction; (ii) show how to use certifiable algorithms to design contracts for and enable self-supervision of learning-based subsystems; (iii) design system monitors that assert the satisfaction of safety requirements or trigger fail-safe procedures in case of failure; (iv) develop a testbed and real demonstrations of certifiable perception on self-driving car data, focusing on key perception functionalities, such as vehicle localization, environment mapping, and object tracking. This research advances the state of the art in CPS and creates a new research field at the boundary between CPS, robotics and autonomous vehicles, computer vision, machine learning, system-level design and runtime verification. Certifiable perception will have a transformative impact on a broad range of autonomous CPS where safety, reliability, security, and accountability are key requirements. These include intelligent transportation, supply chain logistics (e.g., last-mile delivery), new aerospace concepts (e.g., autonomous spacecraft, flying taxis, and drones for national security), service and domestic robotics, and collaborative manufacturing. This impact will be enhanced through the release and dissemination of open-source implementations and teaching material, and via demonstrations on real testbeds. The project also boosts K-12, undergraduate, and graduate education, by supporting and actively engaging students in research activities, and through outreach efforts targeting high school students from underrepresented and underserved communities. This project is in response to the NSF CAREER 20-525 solicitation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Abstract
Luca Carloni
Luca Carloni is an Associate Professor of Computer Science at Columbia University in the City of New York, where he leads the System-Level Design Group. He holds a Laurea Degree Summa cum Laude in Electronics Engineering from the University of Bologna, Italy, a Master of Science in Engineering from the University of California at Berkeley, and a Ph.D. in Electrical Engineering and Computer Sciences from the University of California at Berkeley.
At Berkeley Luca was the 2002 recipient of the Demetri Angelakos Memorial Achievement Award in recognition of altruistic attitude towards fellow graduate students. Luca received the Faculty Early Career Development (CAREER) Award from the National Science Foundation in 2006, was selected as an Alfred P. Sloan Research Fellow in 2008, received the ONR Young Investigator Award in 2010 and the IEEE CEDA Early Career Award in 2012.
His research interests include methodologies and tools for multi-core system-on-chip platforms with emphasis on system-level design and communication synthesis, design and optimization of networks-on-chip, embedded software and distributed embedded systems. Luca coauthored over ninety refereed papers and is the holder of one patent.
Luca is an associate editor of the ACM Transactions in Embedded Computing Systems and the Elsevier Journal of Sustainable Computing. He has served in the technical program committee of several conferences including DAC, DATE, ICCAD, and EMSOFT. In 2010 he served as technical program co-chair of the International Conference on Embedded Software (EMSOFT), the International Symposium on Networks-on-Chip (NOCS), and the International Conference on Formal Methods and Models for Codesign (MEMOCODE).
In 2013 Luca serves as general chair of Embedded Systems Week (ESWeek), the premier event covering all aspects of embedded systems and software.
Luca participates in the Gigascale Systems Research Center (GSRC).
Performance Period: 03/15/2021 - 02/28/2026
Institution: Massachusetts Institute of Technology
Sponsor: National Science Foundation
Award Number: 2044973