Showing papers presented at "Computational Color Imaging Workshop in 2019"

Learning Parametric Functions for Color Image Enhancement

Abstract: In this work we propose a novel CNN-based method for image enhancement that simulates an expert retoucher. The method is fast and accurate at the same time thanks to the decoupling between the inference of the parameters and the color transformation. Specifically, the parameters are inferred from a downsampled version of the raw input image and the transformation is applied to the full resolution input. Different variants of the proposed enhancement method can be generated by varying the parametric functions used as color transformations (i.e. polynomial, piecewise, cosine and radial), and by varying how they are applied (i.e. channelwise or full color). Experimental results show that several variants of the proposed method outperform the state of the art on the MIT-Adobe FiveK dataset.

Evaluating CNN-Based Semantic Food Segmentation Across Illuminants

Abstract: In this paper we aim to explore the potential of Deep Convolutional Neural Networks (DCNNs) on food image segmentation where semantic segmentation paradigm is used to separate food regions from the non-food regions. Specifically, we are interested in evaluating the performance of an efficient DCNN with respect to variability in illumination conditions that can be found in food images taken in real scenarios. To this end we have designed an experimental setup where the network is trained on images rendered as if they were taken under nine different illuminants. We evaluate the food vs. non-food segmentation performance of the network in terms of standard Intersection over Union (IoU) measure. The results of this experimentation are reported and discussed.

Finding a Colour Filter to Make a Camera Colorimetric by Optimisation

Abstract: The Luther condition states that a camera is colorimetric if its spectral sensitivities are a linear transform from the XYZ colour matching functions. Recently, a method has been proposed for finding the optimal coloured filter that when placed in front of a camera, results in effective sensitivities that satisfy the Luther condition. The advantage of this method is that it finds the best filter for all possible physical capture conditions. The disadvantage is that the statistical information of typical scenes are not taken into account.

Web Browsers Colorimetric Characterization

Abstract: The great heterogeneity of mobile display devices currently available on the market makes the implementation of universal color management difficult. To address this problem, we have targeted a common feature of these devices: their ability to run web browsers. In this article we describe a colorimetric characterization tool, uncorrelated to the hardware and operating system used, which is an essential element in the implementation of an universal color management process in web browsers. This tool consists of a software that controls a calorimeter running on a computer located in the same local network as the mobile device running the web browser. It uses colorimetric parameters that allow us to obtain, for the metrics we want to optimize, accurate color transformation models (with maximum errors in \(\varDelta E_{1994}\) and \(\varDelta E_{2000}\) up to 2 times lower than the state of the art).

Reflectance Computation for a Specular Only V-Cavity.

Abstract: The color of a surface structured at the mesoscopic scale differs from the one of a flat surface of the same material because of the light interreflections taking place in the concavities of the surface, as well as the shadowing effect. The color variation depends not only on the surface topology but also on the spectral reflectance of the material, its matte or glossy finishing, and the angular distribution of the incident light. For an accurate prediction of the radiance perceived from each point of the object by an observer or a camera, we must take into account comprehensively the multiple paths of light which can be reflected, scattered or absorbed by the material and its surface. In this paper, we focus on the light reflection component due to the material-air interface, in the special case of a surface structured with parallel, periodical, specular V-shaped ridges, illuminated either by collimated light from any direction of the hemisphere, or by diffuse light. Thanks to an analytical model, we compute the radiance reflected in every direction of the hemisphere by accounting for the different interreflections, according to the angular reflectance of the panels and the aperture angle of the cavity. We can then deduce the apparent reflectance of the cavity when viewed from a large distance.

Evaluation of automatic image color theme extraction methods

Abstract: Color themes are quite important in several fields from visual and graphic art design to image analysis and manipulation. Color themes can be extracted from an image manually by humans or automatically by a software. Plenty of automatic color theme extraction methods, either supervised or unsupervised, have been presented in the state of the art in the last years. Evaluation of a color theme goodness with respect to a reference one is based on visual and subjective comparisons, that may be affected by cultural and social aspects, they are time consuming and not costless. In this paper we experiment several supervised and unsupervised state-of-the-art methods for color theme extraction. To overcome the burden of a subjective evaluation, we experiment the use of a computational metric based on the Earth Mover’s distance for goodness evaluation instead of a subjective one. Results show the best color theme is extracted by using a supervised method based on a regression model trained on user-defined color themes and that the computational metric adopted is comparable to a subjective one.

Improving Generalization Ability of Deep Neural Networks for Visual Recognition Tasks

Abstract: This article discusses generalization ability of convolutional neural networks (CNNs) for visual recognition with special focus on robustness to image degradation. It has been long since CNNs were claimed to surpass human vision, for example, in an object recognition task. However, such claims simply report experimental results that CNNs perform better than humans on a closed set of testing inputs. In fact, CNNs can easily fail for images to which noises are added, when they have not learned the noisy images; this is the case even if humans are barely affected by the added noises. As a solution to this problem, we discuss an approach that first restores the clean image from an input distorted image and then uses it for the target recognition task, where a CNN trained only on clean images is used. For solutions to the first step, we show our recent studies of image restoration. There are multiple different types of image distortion, such as noise, defocus/motion blur, rain-streaks, raindrops, haze etc. We first introduce our recent study of architectural design of CNNs for image restoration targeting at a single, identified type of distortion. We then introduce another study, which proposes to use a single CNN to remove combination of multiple types of distortion with unknown mixture ratio. Although it achieves only lower accuracy than the first method in the case of a single, identified type of distortion, the method will be more useful in practical applications.

Using the Monge-Kantorovitch Transform in Chromagenic Color Constancy for Pathophysiology

Abstract: The Chromagenic color constancy algorithm estimates the light color given two images of the same scene, one filtered and one unfiltered. The key insight underpinning the chromagenic method is that the filtered and unfiltered images are linearly related and that this linear relationship correlates strongly with the illuminant color. In the original method the best linear relationship was found based on the assumption that the filtered and unfiltered images were registered. Generally, this is not the case and implies an expensive image registration step.

Material Appearance Transfer with Visual Cortex Image.

Abstract: Material perception is a current hot topic. Recently a basic research on SHITSUKAN (material perception) has advanced under MEXT (Ministry of Education, Culture, Sports, Science and Technology) in Japan. It is expected to bring innovation for not only traditional craft, ceramic or plastic arts but also more realistic picture displays on 4K/8K HD TVs and VR/CG world. The material perception is said as a phenomenon that our brain feels from retinal images. Now, the analysis is progressing what features of optical images are more strongly related to the stimulus inside the visual cortex of V1–V5.

Physically Plausible Dehazing for Non-physical Dehazing Algorithms

Abstract: Images affected by haze usually present faded colours and loss of contrast, hindering the precision of methods devised for clear images. For this reason, image dehazing is a crucial pre-processing step for applications such as self-driving vehicles or tracking. Some of the most successful dehazing methods in the literature do not follow any physical model and are just based on either image enhancement or image fusion. In this paper, we present a procedure to allow these methods to accomplish the Koschmieder physical model, i.e., to force them to have a unique transmission for all the channels, instead of the per-channel transmission they obtain. Our method is based on coupling the results obtained for each of the three colour channels. It improves the results of the original methods both quantitatively using image metrics, and subjectively via a psychophysical test. It especially helps in terms of avoiding over-saturation and reducing colour artefacts, which are the most common complications faced by image dehazing methods.

On the Acquisition and Reproduction of Material Appearance

Abstract: Currently, new technologies (e.g. 2.5D and 3D printing processes) progress at a fast pace in their capacity to (re)produce an ever-broader range of visual aspects. At the same time, a huge research effort is needed to achieve a comprehensive scientific model for the visual sensations we experience in front of an object in its surrounding. Thanks to the projects MUVApp: Measuring and Understanding Visual Appearance funded by the Research Council of Norway, and ApPEARS: Appearance Printing—European Advanced Research School recently granted by the European Union, significant progress is being made on various topics related with acquisition and reproduction of material appearance, and also on the very understanding of appearance. This paper presents recent, ongoing, and planned research in this exciting field, with a specific emphasis on the MUVApp project.

Conditional Color Gamut for Color Management of Multiview Printed Images

Abstract: A number of printing or surface coloration technologies have recently emerged, able to print color images on various kinds of supports with various benefits in terms of color rendering. Some of them are even able to produce image with variable colors, where each point in the print can display two or more colors according to the illumination or observation conditions (modes). For most printing systems presenting these variable color properties, the colors displayable in one mode depend on the colors that we want to display in the other modes. Because of this interdependence of the colors displayable in the different modes, the color gamut concept as defined for traditional printing is not sufficient anymore to perform efficient color management. We propose to extend it by taking into account the constraints induced by the selection of colors in some modes, and introduce the concept of conditional color gamut, illustrated through the case of recto-verso halftone prints viewed in reflection or transmission modes, where gamuts can be easily predicted thanks to prediction models.

Chromatic Adaptation in Colour Management

Abstract: Chromatic adaptation transforms are used to predict corresponding colours viewed under a different adapting illuminant. In colour management it is often necessary to apply such a transform in order to achieve a corresponding-colour match on a reproduction medium. A linear version of the Bradford CAT has been standardized for this purpose, due to its advantages of computational simplicity and invertibility. Despite being in use in colour management since 2001 the performance of this linear Bradford transform has had limited evaluation. In this paper it is tested for the first time on a comprehensive corresponding-colour data set, and it is shown that the performance is not significantly different from the original Bradford transform, and is a little better than the more recent CAT16 transform. Other issues related to the use of chromatic adaptation in colour management workflows are also discussed.

Haze Transfer Between Images Based on Dark Channel Prior

Abstract: This paper presents a method for transferring haze information between the source and target images. First, we applied the dark channel prior method to the input source and target images. The dark channel prior method was proposed for removing haze from an image. By applying the method thereof, an input image can be decomposed to a scene radiance image, global atmospheric color (haze color) vector, and transmission map. Subsequently, for transferring the haze information, we applied the color transfer method to the decomposed information of source and target images. Finally, we reconstructed a haze-transferred image from the decomposed images. Our results indicate that the proposed haze transfer method can reproduce fine haze-transferred appearance, compared with the input source and target images.

Computational Imaging in Projection Mapping

Abstract: Projection mapping or spatial augmented reality (SAR) has been tremendously widespread over the world. The goal is to seamlessly merge physical and virtual worlds by superimposing computer generated graphics onto real surfaces. In projection mapping applications, target surfaces are generally not suitable for projection. They are textured and non-planar and conventional projectors are specifically designed to display high quality images onto uniformly white and flat surfaces only. Although researchers developed various algorithms to alleviate image quality degradations, the performances were limited by upper bounds resulting from the hardware. I briefly overview the recent advances in the projection mapping research field, particularly focuses on the computational imaging and display approach to overcome technical limitations of current projection hardware in terms of dynamic range, refresh rate, and depth-of-field. I also covers an emerging issue in the projection mapping research, which is dynamic projection mapping. This article is written by reorganizing a previously published state-of-the-art report paper by the same author [13] for an invited talk at the IAPR Computational Color Imaging Workshop (CCIW) 2019.

Evaluating the Material Appearance of Objects Under Different Lighting Distributions Against Natural Illumination

Abstract: The recent development of new solid-state lamps including OLED lighting offered a wide variety of lighting conditions through controlling the spectral power distribution and the spatial distribution of light. The appearance of an object surface is largely influenced by lighting conditions and object materials. Variable control of lighting condition would be useful to offer an optimal material impression. We have investigated the possibility whether the subjective evaluation, comparing material appearance under different lighting distribution with that under natural illumination, is able to determine a lighting condition for an appropriate material appearance. We tested viewing condition consisted of three spotlight sizes and three illuminance levels. Participants chose one viewing condition in which the material appearance of fruits and vegetable food samples was closest to those impression learned from observing and holding freely in a reference natural illumination. In addition, they evaluated impressions of stimuli in each condition by the twelve questionnaires of seven-point scales. The result showed higher tendencies to select the wide spotlight size condition with higher diffuseness of illumination rather than narrower spotlight conditions, suggesting that diffuseness of illumination influences object material appearance. Results of seven-point scales showed differences between samples, but little differences among lighting distribution. It was thus suggested the possibility to provide an optimal lighting condition to offer material appearance similar to material impression learned with visual and tactile information in natural illumination.

Acquisition of 3D Data and Spectral Color by Using RGBD Camera and Programmable Light Source

Abstract: Measurement of shape and color of three-dimensional (3D) object surface is important in many fields including display, digital archives, computer graphics, and computer vision. For instance, the reproduction of object color appearance under various illuminations requires spectral color information of the object surface, and the shape information of the object surface is important for the shading. This paper proposes a method for acquisition of 3D data and spectral color of the object surface by a system consisted of RGBD camera and a programmable light source that can emit narrow band lights. We use Kinect v2 as the RGBD camera. The Kinect has auto white balance and auto contrast, and user cannot control these color compensation functions. Therefore, we estimated the reflection from object in each narrow band image by using achromatic color patches as references. In experiments, we confirmed the estimation accuracy by using achromatic color patches. The results showed that average root mean square error (RMSE) for the spectral color estimation of six color patches was 0.035 in the range in which perfect reflection is defined as 1. We also generated spectral images from 12 band images and reproduced color images under several lighting conditions from the spectral images. Then, we combined the 3D data and the reproduced color image in PLY format. We showed that it is possible to measure 3D data and spectral color of the object by the system consisted of the RGBD camera and the programmable light source.

Spectral Estimation of Chromatically Adapted Corresponding Colors

Abstract: This paper reviews the estimation of spectral reflectance for corresponding colors in XYZ color space, including both corresponding color data sets and chromatically adapted colorimetry. For use in color management workflows, the performance of an inverse transform of the chromatically adapted data was evaluated using spectral estimation. These estimated spectra were then evaluated against the estimated spectral reflectances of reference corresponding color data to analyze the similarity. The results show that established methods using PCA can be used to obtain good spectral estimates, and the methods described in this paper can be implemented in a color managed workflow where spectral processing and output are desired.

An Imaging System for Fourier Coefficients of Spectral Reflectance

Abstract: This paper proposes a system for acquiring the images of complex Fourier coefficients of the spectral reflectance of an object up to the second order at an ordinary frame rate. This feature is realized by a correlation camera and a special illumination called sinusoidally-modulated phase-shift spectral illumination (SMPSSI). The correlation camera produces the temporal correlation between the intensity signal of incident light and external global reference signals pixel by pixel in every frame. The SMPSSI consists of a sum of spectral illuminations sinusoidally modulated at different frequencies with wavelength-linear phase shifts. The proposed system realizes high spectral resolution by performing spectral correlation with sinusoidal reference spectra in the wavelength domain, while maintaining the same temporal and spatial resolution as that of an ordinary video camera. An experimental system is developed with a digital correlation camera and a programmable spectral light source. Experimental results on color guide chips confirm that the proposed system extracted the Fourier coefficients up to the second order accurately.

Makeup Skin Appearance Reproduction by Spectral Projection Mapping

Abstract: Projection mapping can control object surface appearances without painting or finishing. An existing problem in projection mapping is the control of skin appearances. This paper presents a spectral projection mapping method to reproduce makeup skin appearances. Spectral information is significant for the physically accurate reproduction of human skin. First, we propose a spectral projection mapping model for computing the projection spectra and controlling skin appearance. Subsequently, we developed a spectral projection mapping system for realizing skin appearance control. For projecting spatially varying spectra, we used spectral basis functions by non-negative matrix factorization. In our experiment, actual makeup skins and projection-mapped skins were compared spectrally. The results indicated that our method could reproduce makeup skins with good accuracy.

Color-Base Damage Feature Enhanced Support Vector Classifier for Monitoring Quake Image

Abstract: In recent times, significant natural disasters have affected our city lives. This includes the occurrence of large earthquakes that have devastated the city’s infrastructure. During such times of crisis, it is important that emergency response times be as quick as possible to mitigate harm and loss. To achieve this, priority must be given to the various types of initial emergency response. Color image monitoring has the potential to prioritize responses. It uses multi-mode data resources such as openly sourced photos on social media, smartphone cameras, CCTV, and so forth. This paper presents a method to enhance the damaged color features extracted based on a pre-trained deep architecture DenseNet-201 in order to classify damage caused by several earthquakes, whose classifiers are Bayesian optimized to minimize the loss function with cross-validation. This method is applied to a case study of an earthquake image dataset. The study incorporates five target categories, namely bridge collapse, initial smoke and fire, road damage with accident risk to expand secondary loss for relevant users, tsunami damage, and non-disaster. Some advantages have been found when using color feature extraction for monitoring quake damage and further opportunities are remarked (189 words).

A Novel Digital-Camera Characterization Method for Pigment Identification in Cultural Heritage

Abstract: Color represents a primary feature in the field of Art and Cultural Heritage, which can also be of help in defining the conservation state of an artwork. The color identification by means of a digital camera represents a non-destructive methodology which makes use of a non-expensive and portable device and enables a spatial analysis which is not allowed to a colorimeter. The present study compares an original method for camera characterization with two approaches reported in the literature. The comparison is based on parameters such as the Pearson correlation coefficient and the ΔE00 colorimetric difference, computed according to the CIEDE2000 formula. The data sets used for both the “training” and the “validation” processes are (a) the 24 tiles of the Color Checker Passport Photo X-Rite color scale and (b) 30 samples of oil painting laid down on a canvas prepared according to the indications of Giorgio Vasari in his renowned “Le vite”. The data so far available clearly show that our original method leads to results which are similar or better than those furnished by the literature methods.

Colorization of High-Frame-Rate Monochrome Videos Using Synchronized Low-Frame-Rate Color Data

Abstract: The frame rate of a color camera is usually limited by its maximum data bandwidth. To obtain high-frame-rate color videos, we propose to colorize high-frame-rate monochrome videos using data from a system composed of one high-frame-rate monochrome camera and two low-frame-rate color cameras. The cameras are synchronized by external triggering signals. With stereo matching and motion estimation algorithms, colorization of high-frame-rate monochrome videos can be realized. The system is very cost-effective, and the processing steps can be fully automated as demonstrated in the paper.

Long-Term Face Image Analysis Based on Canonical Correlation Analysis on Physical and Psychological Evaluation of Face.

Abstract: In this paper, we analyze the relationship between impression of facial skin and skin pigmentation distribution by applying Canonical Coefficient Analysis (CCA) to multiple physical and psychological features obtained from facial skin. Based on the acquired relationship expression, we modulate the skin pigment features, and appearances of the face with arbitrary psychological features are reproduced. In our previous work, we applied principal component analysis (PCA) to the melanin pigment variation of the facial skin, and we obtained individual differences in it occurring over seven years. In the previous method, as the factor causing individual difference, we considered the frequency of UV care. However, actual skin appearance is thought to depend not only on melanin but also on several other factors. Therefore, in this study, we photographed the faces of women of various ages for 12 years, and at the same time obtained psychological features of appearance. As physical features, melanin and hemoglobin pigmentation distributions, shading and the frequency of UV care for 12 years were obtained. Subjective evaluation values were acquired as psychological features. As a result of CCA, it was found that the whole face can be made lighter in appearance by using UV protection every day continuously for six years or more.

HDR Spectral Video Measurement System.

Abstract: This paper describes a method for measurement of HDR spectral video. In recent years, it is necessary to record images with more accurate color information in various fields such as digital archive, electronic museum, telemedicine, and so on. HDR images and spectral images are one of solutions to the problem. However, in order to acquire these images, it is generally necessary to lengthen the exposure time or replace the filter. Due to these problems, it took time and effort to acquire HDR images and spectral images. So, we propose a system that combines HDR image and spectral image by using four filters and RGB camera. In this paper, after introducing the technology of the previous research, we explain about the proposed method and finally evaluate the result.