Description
To evaluate the image quality deterioration caused by metamerism for digital cameras’ color correction, I collected 43,714,197 spectral reflectance data from several public databases and preprocessed them so that finally 7,326,497 reliable reflectance data were obtained. Since these spectral reflectances practically exist in the real scenes, they provide useful information for the researches of color science and digital image processing, including but not limited to the estimation of metamers, the training sample selection for color adaptation/color correction, the statistical analysis for auto white balance, the calculation of color gamut, etc.
Sources
The total reflectance data were collected from two types of online databases: individual spectral reflectance databases and hyperspectral image databases. Some sources of data are listed as follows (some were downloaded long ago so I can’t remember where I found them).
Spectral reflectance data:
- University of Eastern Finland, Spectral Color Research Group: Spectral Datasets
- Simon Fraser University, Computational Vision Lab: Synthetic Data for Computational Colour Constancy Experiments
- University of Joensuu, Department of Computer Science: Database – Munsell Colors Matt (Spec)
- ISO/TR 16066:2003: Graphic technology – Standard object colour spectra database for colour reproduction evaluation (SOCS)
- More…
Hyperspectral image (HSI) data:
- The University of Manchester, Sensing, Imaging, and Signal Processing Group: Hyperspectral images of natural scenes (link 1, link 2 and link 3)
- Columbia University, Computer Vision Laboratory: Multispectral Image Database
- University of Eastern Finland, Spectral Color Research Group: Joensuu Spectral Image Database
- Stanford University, The Stanford Center for Image Systerms Engineering: Hyperspectral Image Data
- Norwegian University of Science and Technology, Colour and Visual Computing Laboratory: Spectral Image Database for Quality (SIDQ)
- University of East Anglia, The Colour Group: Multispectral Image Database
- University of Granada, Color Imaging Laboratory: UGR Hyperspectral Image Database
- Bristol University, Department of Experimental Psychology: Bristol Hyperspectral Images Database
- Harvard University, School of Engineering and Applied Sciences: Statistics of Real-World Hyperspectral Images
- University of Pennsylvania, Brainard Lab: Hyperspectral Images
- Nanyang Technological University, Dilip K. Prasad: Hyperspectral Images Database
Data processing
First, I preprocessed (downsampling or interpolating) all reflectance data or HSIs so that each spectral reflectances has 31 dimensions: from 400nm to 700nm at 10nm interval. Next, spectral reflectances with too low energy ($\sum{}\mathbf{r} < 0.2$), which might suffer from high noise in the HSI, were discarded. Since the spectral reflectance data extracted from one hyperspectral image usually have high similarity, I then clustered those with “correlation coefficient” higher than 0.99 and remove most of them, only kept those on the vertices of the cluster in the 6-dimension hyperspace, which is the dimensionality reduction result of original 31-dimension hyperspace performed by PCA, on account that the Qhull algorithm is time-consuming when dimensionality is high. More detail about the spectral reflectance data processing will be described in the paper (coming soon).
The correlation coefficient used here is calculated as: \[\mathrm{corr}(\mathbf{r}_1, \mathbf{r}_2) = \frac{(\mathbf{r}_1 – \bar{\mathbf{r}_1})^\mathsf{T}(\mathbf{r}_2 – \bar{\mathbf{r}_2})}{\sqrt{(\mathbf{r}_1 – \bar{\mathbf{r}_1})^\mathsf{T}(\mathbf{r}_1 – \bar{\mathbf{r}_1})}\cdot\sqrt{(\mathbf{r}_2 – \bar{\mathbf{r}_2})^\mathsf{T}(\mathbf{r}_2 – \bar{\mathbf{r}_2})}}\,,\]
where $\bar{\mathbf{r}_i} = \frac{1}{n}\sum^n_j{}\mathrm{r}_{ij}$ and $n = 31$.
The distribution of chromaticities of all reflectance spectra (under CIE D65) before and after data processing are illustrated in Fig.1. It can be noticed that large amount of spectral reflectances on the periphery of xy diagram were removed, bacause these data usually correspond to the dark “pixel” in the HSI and their colors are biased by the high spectral noise.
For the sake of storage size, the data were converted from double to single precision in MATLAB.
Download
You may use this database for research purposes on condition that you acknowledge the source.
- Total database containing 7,326,497 reflectance spectra: TotalRefs.mat (625M)
- A compact version containing only 114,120 individual reflectance spectra is also available: TotalRefs_IndividualSpectra.mat (11.1M)
The distribution of chromaticities of this compact version (under CIE D65) is shown in Fig.2.
