Single and Multiple Illuminant Estimation Using Convolutional Neural Networks

Abstract

In this paper we present a three-stage method for the estimation of the color of the illuminant in RAW images. The first stage uses a Convolutional Neural Network that has been specially designed to produce multiple local estimates of the illuminant. The second stage, given the local estimates, determines the number of illuminants in the scene. Finally, local illuminant estimates are refined by non-linear local aggregation, resulting in a global estimate in case of single illuminant. An extensive comparison with both local and global illuminant estimation methods in the state of the art, on standard datasets with single and multiple illuminants, proves the effectiveness of our method.

Existing System

Kernel Density Estimator (KDE), Support Vector Regression (SVR).

Proposed System

In this work we have developed a CNN-based color constancy algorithm that combines feature learning and regression as a complete optimization process, which enables us to employ modern training techniques to boost performance. The network has been specially designed to work on image patches in order to estimate the local illuminant color. Local estimates are then refined by local non-linear regression. The size of the support of the regression is automatically determined by a multiple illuminant detector: if only a single illuminant is detected, the support size is the whole image and a global illuminant estimate is produced. Since one of the assumptions that is often violated in color constancy is the presence of a uniform illumination in the scene, we have extended the applicability of the proposed algorithm to the case of non-uniform illumination. The method is adaptive, being able to distinguish and process in different ways images of scenes taken under a uniform and those acquired under non-uniform illumination. In the case of uniform illumination, the multiple local estimates must be aggregated in a single global estimate. To do so we designed a new local regression method that replaces the per-channel median operator with a non-linear mapping based on a RBF kernel over local statistics of the CNN estimates. The mapping is optimized by applying a regression procedure that minimizes the median angular error on the training set. In case of non-uniform illumination, the local estimates are refined using the same local regression method applied for uniform illumination, changing its support size on the basis of the spatial extent of the illuminants in the scene.

Architecture

BLOCK DIAGRAM