This paper presents a new reflectance model for rendering computer synthesized images. The model accounts for the relative brightness of different materials and light sources in the same scene. It describes the directional distribution of the reflected light and a color shift that occurs as the reflectance changes with incidence angle. The paper presents a method for obtaining the spectral energy distribution of the light reflected from an object made of a specific real material and discusses a procedure for accurately reproducing the color associated with the spectral energy distribution. The model is applied to the simulation of a metal and a plastic.

A reflectance model for computer graphics

In this paper we present Pupil -- an accessible, affordable, and extensible open source platform for pervasive eye tracking and gaze-based interaction Pupil comprises 1) a light-weight eye tracking headset, 2) an open source software framework for mobile eye tracking, as well as 3) a graphical user interface to playback and visualize video and gaze data Pupil features high-resolution scene and eye cameras for monocular and binocular gaze estimation The software and GUI are platform-independent and include state-of-the-art algorithms for real-time pupil detection and tracking, calibration, and accurate gaze estimation Results of a performance evaluation show that Pupil can provide an average gaze estimation accuracy of 06 degree of visual angle (008 degree precision) with a processing pipeline latency of only 0045 seconds

Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction

Demosaicking and denoising are the key first stages of the digital imaging pipeline but they are also a severely ill-posed problem that infers three color values per pixel from a single noisy measurement. Earlier methods rely on hand-crafted filters or priors and still exhibit disturbing visual artifacts in hard cases such as moire or thin edges. We introduce a new data-driven approach for these challenges: we train a deep neural network on a large corpus of images instead of using hand-tuned filters. While deep learning has shown great success, its naive application using existing training datasets does not give satisfactory results for our problem because these datasets lack hard cases. To create a better training set, we present metrics to identify difficult patches and techniques for mining community photographs for such patches. Our experiments show that this network and training procedure outperform state-of-the-art both on noisy and noise-free data. Furthermore, our algorithm is an order of magnitude faster than the previous best performing techniques.

Deep joint demosaicking and denoising

The Laplacian pyramid is ubiquitous for decomposing images into multiple scales and is widely used for image analysis. However, because it is constructed with spatially invariant Gaussian kernels, the Laplacian pyramid is widely believed as being unable to represent edges well and as being ill-suited for edge-aware operations such as edge-preserving smoothing and tone mapping. To tackle these tasks, a wealth of alternative techniques and representations have been proposed, e.g., anisotropic diffusion, neighborhood filtering, and specialized wavelet bases. While these methods have demonstrated successful results, they come at the price of additional complexity, often accompanied by higher computational cost or the need to post-process the generated results. In this paper, we show state-of-the-art edge-aware processing using standard Laplacian pyramids. We characterize edges with a simple threshold on pixel values that allows us to differentiate large-scale edges from small-scale details. Building upon this result, we propose a set of image filters to achieve edge-preserving smoothing, detail enhancement, tone mapping, and inverse tone mapping. The advantage of our approach is its simplicity and flexibility, relying only on simple point-wise nonlinearities and small Gaussian convolutions; no optimization or post-processing is required. As we demonstrate, our method produces consistently high-quality results, without degrading edges or introducing halos.

/pdf/local-laplacian-filters-edge-aware-image-processing-with-a-1zvb7m6n5q.pdf

Local Laplacian filters: edge-aware image processing with a Laplacian pyramid

This paper provides a classification of perceptual issues in augmented reality, created with a visual processing and interpretation pipeline in mind. We organize issues into ones related to the environment, capturing, augmentation, display, and individual user differences. We also illuminate issues associated with more recent platforms such as handhelds or projector-camera systems. Throughout, we describe current approaches to addressing these problems, and suggest directions for future research.

https://web.cse.msstate.edu/~swan/publications/papers/2010_Kruijff-etal_Perceptual-Issues_ISMAR.pdf

Perceptual issues in augmented reality revisited

To provide a convincing proof that a new method is better than the state of the art, computer graphics projects are often accompanied by user studies, in which a group of observers rank or rate results of several algorithms. Such user studies, known as subjective image quality assessment experiments, can be very time-consuming and do not guarantee to produce conclusive results. This paper is intended to help design efficient and rigorous quality assessment experiments and emphasise the key aspects of the results analysis. To promote good standards of data analysis, we review the major methods for data analysis, such as establishing confidence intervals, statistical testing and retrospective power analysis. Two methods of visualising ranking results together with the meaningful information about the statistical and practical significance are explored. Finally, we compare four most prominent subjective quality assessment methods: single-stimulus, double-stimulus, forced-choice pairwise comparison and similarity judgements. We conclude that the forced-choice pairwise comparison method results in the smallest measurement variance and thus produces the most accurate results. This method is also the most time-efficient, assuming a moderate number of compared conditions. © 2012 Wiley Periodicals, Inc.

/pdf/comparison-of-four-subjective-methods-for-image-quality-3nbtzkggta.pdf

Comparison of Four Subjective Methods for Image Quality Assessment

Tone mapping algorithms offer sophisticated methods for mapping a real-world luminance range to the luminance range of the output medium but they often cause changes in color appearance. In this work we conduct a series of subjective appearance matching experiments to measure the change in image colorfulness after contrast compression and enhancement. The results indicate that the relation between contrast compression and the color saturation correction that matches color appearance is non-linear and smaller color correction is required for small change of contrast. We demonstrate that the relation cannot be fully explained by color appearance models. We propose color correction formulas that can be used with existing tone mapping algorithms. We extend existing global and local tone mapping operators and show that the proposed color correction formulas can preserve original image colors after tone scale manipulation.

/pdf/color-correction-for-tone-mapping-4l6snu6gtx.pdf

Color correction for tone mapping

We present a fully automatic method for eliminating misalig nments between a sequence of hand-held photographs taken at different exposures. The key component of the techn ique is the SIFT method that is employed to search for key-points (or feature-points) in consecutive images. The key-points are used to find matrices, that transform a set of images to a single coordinate system and eliminate an y global misalignments (including general planar homography). We employ this technique to capture high dynam ic range images from a set of photographs taken at different exposures, where misalignments can cause blurri ng and artifacts, and prevent achieving high quality HDR images. The proposed alignment technique works well for ove and under-exposed images and is not sensitive to an image content. We present our implementation of the tec hnique and the results of tests made for variety of photographs.

/pdf/image-registration-for-multi-exposure-high-dynamic-range-4tym59cxpm.pdf

Image Registration for Multi-exposure High Dynamic Range Image Acquisition

The advances in high dynamic range (HDR) imaging, especially in the display and camera technology, have a significant
impact on the existing imaging systems. The assumptions of the traditional low-dynamic range imaging, designed for
paper print as a major output medium, are ill suited for the range of visual material that is shown on modern displays. For
example, the common assumption that the brightest color in an image is white can be hardly justified for high contrast
LCD displays, not to mention next generation HDR displays, that can easily create bright highlights and the impression
of self-luminous colors. We argue that high dynamic range representation can encode images regardless of the technology
used to create and display them, with the accuracy that is only constrained by the limitations of the human eye and
not a particular output medium. To facilitate the research on high dynamic range imaging, we have created a software
package (http://pfstools.sourceforge.net/) capable of handling HDR data on all stages of image and video processing. The
software package is available as open source under the General Public License and includes solutions for high quality
image acquisition from multiple exposures, a range of tone mapping algorithms and a visual difference predictor for HDR
images. Examples of shell scripts demonstrate how the software can be used for processing single images as well as video
sequences.

High Dynamic Range Imaging Pipeline : Perception-Motivated Representation of Visual Content

Current rendering techniques struggle to fulfill quality and power efficiency requirements imposed by new display devices such as virtual reality headsets. A promising solution to overcome these problems is foveated rendering, which exploits gaze information to reduce rendering quality for the peripheral vision where the requirements of the human visual system are significantly lower. Most of the current solutions model the sensitivity as a function of eccentricity, neglecting the fact that it also is strongly influenced by the displayed content. In this work, we propose a new luminance-contrast-aware foveated rendering technique which demonstrates that the computational savings of foveated rendering can be significantly improved if local luminance contrast of the image is analyzed. To this end, we first study the resolution requirements at different eccentricities as a function of luminance patterns. We later use this information to derive a low-cost predictor of the foveated rendering parameters. Its main feature is the ability to predict the parameters using only a low-resolution version of the current frame, even though the prediction holds for high-resolution rendering. This property is essential for the estimation of required quality before the full-resolution image is rendered. We demonstrate that our predictor can efficiently drive the foveated rendering technique and analyze its benefits in a series of user experiments.

/pdf/luminance-contrast-aware-foveated-rendering-2ssopfc8e9.pdf

Radosław Mantiuk

Papers

Comparison of Four Subjective Methods for Image Quality Assessment

Color correction for tone mapping

Image Registration for Multi-exposure High Dynamic Range Image Acquisition

High Dynamic Range Imaging Pipeline : Perception-Motivated Representation of Visual Content

Luminance-contrast-aware foveated rendering