Real-Time Computer Vision in Autonomous Vehicles: Real Fast Isn't Good Enough

A significant gap exists between how computer-vision software and embedded safety-critical software define the term real time. In computer vision, "real time" often means "real fast" and refers to high average throughput with low latency. In contrast, safety-critical embedded systems consider "real time" to be a statement about predictable timing under continuous performance. Put in a larger context, this disparity in definitions is merely a symptom of the fact that, historically, computer vision and embedded systems have evolved independently. With the advent of autonomous vehicles, where safety depends on both computer vision and predictable timing, a clear separation between vision and embedded software is no longer possible.

In the near future, autonomous vehicles will face the need for real-time certification, which will cause the "real-time" vs. "real-fast" disconnect to become more problematic as time passes. Certifying applications that are merely "real-fast" will be almost impossible, but, as of now, there is no simple way to map traditional computer-vision applications into the frameworks needed to ensure predictable timing required by safety-critical systems. Our project seeks to address this problem by developing a framework to bridge this gap between "real-fast" and "real- time" software.

This project focuses on four principal objectives:
•    Develop a real-time-aware computer-vision API.
•    Develop real-time schedulability analysis targeting our new API.
•    Develop real-time computer-vision algorithms that exploit our API's new features.
•    Experimentally compare "real-fast" and "real-time" computer vision.