MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

Computer and Robot Vision

Color science is a multidisciplinary field with broad applications in industries such as digital imaging, coatings and textiles, food, lighting, archiving, art, and fashion. Accurate definition and measurement of color appearance is a challenging task that directly affects color reproduction in such applications. Color Appearance Models addresses those challenges and offers insight into the preferred solutions. Extensive research on the human visual system (HVS) and color vision has been performed in the last century, and this book contains a good overview of the most important and relevant literature regarding color appearance models.

https://www.spiedigitallibrary.org/journals/Journal-of-Electronic-Imaging/volume-23/issue-2/029901/Color-Appearance-Models/10.1117/1.JEI.23.2.029901.pdf

Color Appearance Models

Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data.

/pdf/light-field-image-processing-an-overview-3jq61afb1y.pdf

Light Field Image Processing: An Overview

Color as a Science

In this paper the authors propose a method to use interpolation to eliminate characters in only one image with telops or other text through an image interpolation method that uses the eigenspace method. Background scenes and other images have a fractal character, and often the self-correlation in the image can be assumed to be high. The authors focus on this point and represent rules for describing the image based on an eigenspace consisting of only one image that has defects. The eigenspace generated in this manner reflects the features of the image, and by using this eigenspace, image interpolation can be achieved. Although this interpolation method does not restore the original image, the authors confirmed through experimental results that it can provide interpolation without a feeling of oddness for images which have a high level of self-correlation. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 38(1): 87– 96, 2007; Published online in Wiley InterScience (). DOI 10.1002sscj.10319.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/scj.10319

Image interpolation using BPLP method on the eigenspace

The fundamental issues in Augmented Reality (AR) are on how to naturally mediate the reality with virtual content as seen by users In AR applications with Optical See-Through Head-Mounted Displays (OST-HMD), the issues often raise the problem of rendering color on the OST-HMD consistently to input colors However, due to various display constraints and eye properties, it is still a challenging task to indistinguishably reproduce the colors on OST-HMDs An approach to solve this problem is to pre-process the input color so that a user perceives the output color on the display to be the same as the input We propose a color calibration method for OST-HMDs We start from modeling the physical optics in the rendering and perception process between the HMD and the eye We treat the color distortion as a semi-parametric model which separates the non-linear color distortion and the linear color shift We demonstrate that calibrated images regain their original appearance on two OST-HMD setups with both synthetic and real datasets Furthermore, we analyze the limitations of the proposed method and remaining problems of the color reproduction in OST-HMDs We then discuss how to realize more practical color reproduction methods for future HMD-eye system

/pdf/semi-parametric-color-reproduction-method-for-optical-see-15kdpb33ii.pdf

Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays

We present a display for optical see-through near-eye displays based on light attenuation, a new paradigm that forms images by spatially subtracting colors of light Existing optical see-through head-mounted displays (OST-HMDs) form virtual images in an additive manner—they optically combine the light from an embedded light source such as a microdisplay into the users' field of view (FoV) Instead, our light attenuation display filters the color of the real background light pixel-wise in the users' see-through view, resulting in an image as a spatial color filter Our image formation is complementary to existing light-additive OST-HMDs The core optical component in our system is a phase-only spatial light modulator (PSLM), a liquid crystal module that can control the phase of the light in each pixel By combining PSLMs with polarization optics, our system realizes a spatially programmable color filter In this paper, we introduce our optics design, evaluate the spatial color filter, consider applications including image rendering and FoV color control, and discuss the limitations of the current prototype

Light Attenuation Display: Subtractive See-Through Near-Eye Display via Spatial Color Filtering

We introduce a new projection display technique that converts a visual material appearance of target object. Unlike conventional projection display, our approach allowed us successive material appearance manipulation by the projector camera feedback without scene modeling. First, we introduce an appearance control framework with a coaxial projector camera system. Next, we introduce two image based material appearance manipulation methods of translucency and glossiness. Last, we verify the ability of the material appearance manipulation of the proposed display technique through the experiments.

/pdf/projection-based-real-time-material-appearance-manipulation-4cuhhd4yaj.pdf

Projection Based Real-Time Material Appearance Manipulation

The unified technique for the irradiance correction and appearance enhancement for the real scene is proposed in this paper. The proposed method employed MPC algorithm for the projector camera system and enabled arbitrary appearance control such like photo retouching software in the real world. In the experiment, the appearance control of saturation enhancement, color removal, phase control, edge enhancement, image blur, makes unique brightness and the other enhancements for the real scene are shown.

/pdf/appearance-control-using-projection-with-model-predictive-l1ia9m0mrx.pdf

Toshiyuki Amano

Papers

Image interpolation using BPLP method on the eigenspace

Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays

Light Attenuation Display: Subtractive See-Through Near-Eye Display via Spatial Color Filtering

Projection Based Real-Time Material Appearance Manipulation

Appearance Control Using Projection with Model Predictive Control