Description

For digital imaging devices, the image signal processing (ISP) is a workflow that transforms the raw data, captured by the sensor, to the final output image file which can be transfered and displayed among different media. A generalized ISP pipeline involves A/D conversion, linearization, demosaicking, pre-processing, white balance correction, color correction, gamma correction, image enhancement, output encoding, etc.

In this project, we proposed an on-chip ISP solution for mobile devices, including camera spectral sensitivity estimation, auto white balance (AWB) algorithm, color lens shading correction (LSC) and color correction (CC). Due to confidentiality agreement and commercial purposes, it is not allowed to publish anything more than a table of contents or some trivial figures here.

Figures

**Figure 1.** The distribution of 273 typical illuminants on the “particular” X-Y plane.

**Figure 2.** Estimated camera spectral sensitivities using different modulation factor $\kappa$.

**Figure 3.** Compensations for lens shadings (vignettings) of the green channel under different illuminants.

**Figure 4.** 2-D histogram of neutral pixels on the “particular” X-Y plane.

**Figure 5.** Standard gamut for one camera model under D65 illuminant.

**Figure 7.** 2-D gamut mapping AWB algorithm. (see here for implementation details)

**igure 8.** Corresponding AWB gains for different illuminants.

Instantiations

In this section, four scenarios captured by a tablet prototype are showed to illustrate how the ISP pipeline works. All images below have been down-sampled for the sake of page’s loading speed. Click on images to zoom in.

Instantiation 1: color checker

Instantiation 2: typical outdoor scene

Instantiation 3: monochromatic scene: container

Instantiation 4: monochromatic scene: lawn

Results

In this section, we show several image pairs of the same scenes photographed by iPhone 6S and our ISP pipeline. Guess which column corresponds the outputs by our algorithm?

Scene 1: office with compounded illuminants

Scene 2: Audi

Scene 3: notice board

Scene 4: thicket

Scene 5: external wall of building in the night

Scene 6: close-up of flower

Reference

A. Gijsenij, T. Gevers and J. Weijer, “Generalized Gamut Mapping using Image Derivative Structures for Color Constancy,” International Journal of Computer Vision 86, pp. 127–139 (2010).
G. D. Finlayson, “Color Correction Using Root-Polynomial Regression,” IEEE Transactions on Image Processing, 24, pp. 1460-1470 (2015).
T. Tajbakhsh, “Color lens shade compensation achieved by linear regression of piece-wise bilinear spline functions,” Proc. SPIE, 7537 (2010).
D. A. Forsyth, “A novel algorithm for color constancy,” International Journal of Computer Vision, 5, pp. 5–35 (1990).
E. Fainstain, “Low noise color correction matrix function in digital image capture systems and methods,” US Patent, US7868928 B2, Samsung Electronics Co., Ltd. (2011).
More…

Feel free to contact me with any suggestions/corrections/comments.

projects

Rockchip on-chip auto white balance, lens shading correction and color correction image signal processing (ISP) pipeline

Description

Figures