Smart Flexible Camera Sheet: Ultra-thin Semantic-guided Cooperative Micro-camera Array

Abstract:

This highly interdisciplinary research addresses two fundamental challenges in image sensing and image understanding: 1) versatile camera systems in a small form factor, and 2) 3-dimensional scene and object recognition from 2-dimensional photos. These fundamental challenges are tackled together by developing a cyber-physical imaging system, called a smart flexible camera sheet, which integrates an array of many micro-cameras (each millimeters in size) onto a thin substrate. The substrate has a flexible geometric shape and the orientation of each camera is individually adjusted and controlled in real time via intelligent algorithms. The overall imaging system is ultra-thin and space-efficient, and can be easily mounted onto or embedded into any planar or curved surface. Hence it opens up a plethora of new civilian and military applications where surveillance and visual monitoring are required, thus bearing great commercialization potential. Examples of applications include: smart vehicles, smart transportation, highway safety, smart civil infrastructure, manufacturing lines, battlefield surveillance and reconnaissance, sensor networks, mobile robotics, medical facilities, and patient care.

We report new progress in the development of tunable, electrically-driven lenses for use in our micro-camera array as well as flexible transmission lines for CMOS sensor data capture on the backside of our flexible camera sheet substrate. We have developed and fabricated an artificial light collection array in order to improve raw image quality. Additionally, we report a novel multi-view image noise reduction algorithm for the camera array using a data structure called 3D focus image stacks.