Smart Flexible Camera Sheet: Ultra-thin Semantic-guided Cooperative Micro-camera Array

Abstract:

Our proposed research is motivated by two fundamental challenges that are traditionally studied in separate disciplines. The first key challenge is that in existing digital cameras, the functionalities of a camera are constrained by its size due to the optics and quantum nature of light. It remains a challenge to design a camera architecture that has both - a small form factor and has versatile functionalities, e.g. high signal-to-noise ratio and optical zoom, multiple planes of focus and simultaneous capture of both peripheral and foveal vision. The second challenge lies in the field of computer vision - object recognition. Although significant progress has been made in recognition of specialized object classes such as text, faces etc., generic object recognition and scene understanding from images and videos remains a fundamental challenge. We propose a unified solution to address both these challenges. In particular, we propose to develop a cyber- physical imaging system that integrates a micro-camera array on a ‘smart’ flexible sheet. Image understanding algorithms will control how the cameras, individually or collectively, target and track objects at different angles and distances. Such image sensing from multiple cameras inherently contains depth information of the underlying 3D scene. This depth information can provide information about object size, background separation and occlusion relationship between objects – which in turn will enhance image understanding. Till date, our research has focused on the fabrication of variable-focus lenses for the micro- cameras, development of light concentrators for enhanced collection and concentration of light to compensate for small lens apertures, and the development of algorithms to combine noisy image data from multiple cameras into a single sharp image.