Depth Map Super-Resolution Considering View Synthesis Quality

Abstract

Accurate and high-quality depth maps are required in lots of 3D applications, such as multi-view rendering, 3D reconstruction and 3DTV. However, the resolution of captured depth image is much lower than that of its corresponding color image, which affects its application performance. In this paper, we propose a novel depth map super-resolution (SR) method by taking view synthesis quality into account. The proposed approach mainly includes two technical contributions. First, since the captured low-resolution (LR) depth map may be corrupted by noise and occlusion, we propose a credibility based multiview depth maps fusion strategy, which considers the view synthesis quality and interview correlation, to refine the LR depth map. Second, we propose a view synthesis quality based trilateral depth-map up-sampling method, which considers depth smoothness, texture similarity and view synthesis quality in the up-sampling filter.

Existing System

Laser-Scanning Method, Depth Image Based Rendering (Dibr), Super-Resolution (SR) Technique.

Proposed System

A novel depth map up-sampling method is proposed which considers both the view synthesis quality and the interview correlation between LR depth maps. There are two key technical contributions in this paper. First, since the captured multi-view depth maps are not consistent with each other due to noise and occlusion, we propose a credibility based multi-view depth maps fusion strategy, which considers the view synthesis quality and the interview correlation between LR depth maps. Second, we propose a view synthesis quality based trilateral depth-map upsampling method, which considers depth smoothness, texture similarity and synthesized view quality in the process of upsampling. We take view synthesis quality into consideration in both multi-view depth fusion and depth up-sampling processes. In the MDMF process, we obtain a refined LR depth map by introducing the view synthesis quality into the judgement of depth value credibility. In the depth up-sampling process, we propose a VSQ-TDU method, which introduces depth smoothness, texture similarity and synthesized view quality into the up-sampling filter.

Architecture

BLOCK DIAGRAM