Visible to the public The HCI Stereo Metrics: Geometry-Aware Performance Analysis of Stereo Algorithms

TitleThe HCI Stereo Metrics: Geometry-Aware Performance Analysis of Stereo Algorithms
Publication TypeConference Paper
Year of Publication2015
AuthorsHonauer, K., Maier-Hein, L., Kondermann, D.
Conference Name2015 IEEE International Conference on Computer Vision (ICCV)
Date Publisheddec
KeywordsAlgorithm design and analysis, Benchmark testing, computational geometry, depth discontinuities, edge fattening, fine geometric structures, geometry-aware performance analysis, ground truth disparity maps, HCI stereo metrics, human computer interaction, mean square error methods, Middlebury dataset, multidimensional performance evaluation, object detection, performance evaluation, planar surfaces, pubcrawl170111, radar charts, root mean squared error, semantic cues, stacked bar charts, stereo algorithms, stereo image processing, Surface reconstruction, surface smoothness

Performance characterization of stereo methods is mandatory to decide which algorithm is useful for which application. Prevalent benchmarks mainly use the root mean squared error (RMS) with respect to ground truth disparity maps to quantify algorithm performance. We show that the RMS is of limited expressiveness for algorithm selection and introduce the HCI Stereo Metrics. These metrics assess stereo results by harnessing three semantic cues: depth discontinuities, planar surfaces, and fine geometric structures. For each cue, we extract the relevant set of pixels from existing ground truth. We then apply our evaluation functions to quantify characteristics such as edge fattening and surface smoothness. We demonstrate that our approach supports practitioners in selecting the most suitable algorithm for their application. Using the new Middlebury dataset, we show that rankings based on our metrics reveal specific algorithm strengths and weaknesses which are not quantified by existing metrics. We finally show how stacked bar charts and radar charts visually support multidimensional performance evaluation. An interactive stereo benchmark based on the proposed metrics and visualizations is available at:

Citation Keyhonauer_hci_2015