HYPERSPECTRAL IMAGE SUPER-RESOLUTION WITH A MOSAIC RGB IMAGE

Abstract

Recently, many hyperspectral (HS) image super resolution methods that merge a low spatial resolution HS image and a high spatial resolution three-channel RGB image have been proposed in spectral imaging. A largely ignored fact is that most existing commercial RGB cameras capture high resolution images by a single CCD/CMOS sensor equipped with a color filter array (CFA). In this paper, we account for the common imaging mechanism of commercial RGB cameras, and propose to use a mosaic RGB image for HS image super-resolution, which prevents demosaicing error and thus its propagation into the HS image super-resolution results. We design a proper nonlocal low-rank regularization to exploit the intrinsic properties — rich self-repeating patterns and high correlation across spectra — within HS images of natural scenes, and formulate the HS image super-resolution task into a variational optimization problem, which can be efficiently solved via the alternating direction method of multipliers (ADMM). The effectiveness of the proposed method has been evaluated on two benchmark datasets, demonstrating that the proposed method can provide substantial improvement over the current state-of-the-art HS image super resolution methods without considering the mosaicing effect. Finally, we show that our method can also perform well in the real capture system.

Existing System

Alternating direction method of multipliers (ADMM), Hierarchical Beta process with Gaussian process prior.

Proposed System

In this paper, we propose a non-local low-rank approximation based method for HS image super-resolution by using a mosaic RGB image. The non-local self-similarity is utilized to describe the intrinsic properties of HS images rich self-repeating patterns and high correlation across spectra via non-local low-rank approximation. Specifically, to take into account the non-local self-similarity in both space and spectrum, we group a set of similar cubic patches for each exemplar patch and make the non-local low-rank regularization on this set, while the cubic patch implicitly includes the correlation across spectra. This low rank regularization and the input measurements are involved into a unified variational optimization model, which can be efficiently solved via the alternating direction method of multipliers (ADMM). The effectiveness of our method is demonstrated on two benchmark datasets, which outperforms the current state-of-the-art hybrid camera systems based HS image super-resolution methods in terms of both objective metric and subjective visual quality. In summary, our main contributions are that we 1) Propose to use a mosaic RGB image for HS image super-resolution, which is more relevant to the imaging mechanism of most commercial RGB cameras; 2) Avoid error propagation from the demosaicing process, which undermines existing HS image super-resolution methods with a full three-channel RGB image; 3) Exploit the intrinsic properties of HS images to design a model and develop an iterative numerical algorithm to efficiently solve the proposed model.

Architecture

BLOCK DIAGRAM