|Depth Enhancement via Low-rank Matrix Completion|
|Si Lu1, Xiaofeng Ren2, and Feng Liu1|
of Computer Science, Portland State University
2Department of Computer Science and Engineering, University of Washington
|Depth captured by consumer RGB-D cameras is often noisy and misses values at some pixels, especially around object boundaries. Most existing methods complete the missing depth values guided by the corresponding color image. When the color image is noisy or the correlation between color and depth is weak, the depth map cannot be properly enhanced. In this paper, we present a depth map enhancement algorithm that performs depth map completion and de-noising simultaneously. Our method is based on the observation that similar RGB-D patches lie in a very low-dimensional subspace. We can then assemble the similar patches into a matrix and enforce this low-rank subspace constraint. This low-rank subspace constraint essentially captures the underlying structure in the RGB-D patches and enables robust depth enhancement against the noise or weak correlation between color and depth. Based on this subspace constraint, our method formulates depth map enhancement as a low-rank matrix completion problem. Since the rank of a matrix changes over matrices, we develop a data-driven method to automatically determine the rank number for each matrix. The experiments on both public benchmark and our own captured RGB-D images show that our method can effectively enhance depth maps.|
Xiaofeng Ren, and Feng Liu. Depth
Enhancement via Low-rank Matrix Completion.
IEEE CVPR 2014. PDF
We apply our method on
images from three datasets, namely the Middlebury dataset [1, 2, 3, 4], the
RGBZ dataset  and our own RGBD dataset captured by ASUS Xtion Pro.
You can download our data as a zip package or click the links below for details
 H. Hirschmuller and D.
Scharstein. Evaluation of cost functions for stereo matching. In IEEE CVPR,
 D. Scharstein and C. Pal. Learning conditional random fields for stereo. In IEEE CVPR, 2007.