Showing papers by "Wolfgang Heidrich published in 2010"

High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting

Abstract: Foreword Preface 1 Introduction 2 Light And Color 3 HDR Image Encodings 4 HDR Image Capture 5 Display Devices 6 The Human Visual System and HDR Tone Mapping 7 Spatial Tone Reproduction 8 Frequency Domain and Gradient Domain Tone Reproduction 9 Image-Based Lighting List of Symbols References Index

High resolution passive facial performance capture

Abstract: We introduce a purely passive facial capture approach that uses only an array of video cameras, but requires no template facial geometry, no special makeup or markers, and no active lighting. We obtain initial geometry using multi-view stereo, and then use a novel approach for automatically tracking texture detail across the frames. As a result, we obtain a high-resolution sequence of compatibly triangulated and parameterized meshes. The resulting sequence can be rendered with dynamically captured textures, while also consistently applying texture changes such as virtual makeup.

Transparent and Specular Object Reconstruction

Abstract: This state of the art report covers reconstruction methods for transparent and specular objects or phenomena. While the 3D acquisition of opaque surfaces with Lambertian reflectance is a well-studied problem, transparent, refractive, specular and potentially dynamic scenes pose challenging problems for acquisition systems. This report reviews and categorizes the literature in this field. Despite tremendous interest in object digitization, the acquisition of digital models of transparent or specular objects is far from being a solved problem. On the other hand, real-world data is in high demand for applications such as object modelling, preservation of historic artefacts and as input to data-driven modelling techniques. With this report we aim at providing a reference for and an introduction to the field of transparent and specular object reconstruction. We describe acquisition approaches for different classes of objects. Transparent objects/phenomena that do not change the straight ray geometry can be found foremost in natural phenomena. Refraction effects are usually small and can be considered negligible for these objects. Phenomena as diverse as fire, smoke, and interstellar nebulae can be modelled using a straight ray model of image formation. Refractive and specular surfaces on the other hand change the straight rays into usually piecewise linear ray paths, adding additional complexity to the reconstruction problem. Translucent objects exhibit significant sub-surface scattering effects rendering traditional acquisition approaches unstable. Different classes of techniques have been developed to deal with these problems and good reconstruction results can be achieved with current state-of-the-art techniques. However, the approaches are still specialized and targeted at very specific object classes. We classify the existing literature and hope to provide an entry point to this exiting field.

CALTag: High Precision Fiducial Markers for Camera Calibration

Abstract: We present a self-identifying marker pattern for camera calibration, together with the associated detection algorithm. The pattern is designed to support high-precision, fully-automatic localization of calibration points, as well as identification of individual markers in the presence of significant occlusions, uneven illumination, and observations under extremely acute angles. The detection algorithm is efficient and free of parameters. After calibration we obtain reprojection errors significantly lower than with state-of-the art self-identifying reference patterns.

A Basis Illumination Approach to BRDF Measurement

Abstract: Realistic descriptions of surface reflectance have long been a topic of interest in both computer vision and computer graphics research. In this paper, we describe a novel high speed approach for the acquisition of bidirectional reflectance distribution functions (BRDFs). We develop a new theory for directly measuring BRDFs in a basis representation by projecting incident light as a sequence of basis functions from a spherical zone of directions. We derive an orthonormal basis over spherical zones that is ideally suited for this task. BRDF values outside the zonal directions are extrapolated by re-projecting the zonal measurements into a spherical harmonics basis, or by fitting analytical reflection models to the data. For specular materials, we experiment with alternative basis acquisition approaches such as compressive sensing with a random subset of the higher order orthonormal zonal basis functions, as well as measuring the response to basis defined by an analytical model as a way of optically fitting the BRDF to such a representation. We verify this approach with a compact optical setup that requires no moving parts and only a small number of image measurements. Using this approach, a BRDF can be measured in just a few minutes.

Globally consistent space-time reconstruction

Abstract: Most objects deform gradually over time, without abrupt changes in geometry or topology, such as changes in genus. Correct space-time reconstruction of such objects should satisfy this gradual change prior. This requirement necessitates a globally consistent interpretation of spatial adjacency. Consider the capture of a surface that comes in contact with itself during the deformation process, such as a hand with different fingers touching one another in parts of the sequence. Naive reconstruction would glue the contact regions together for the duration of each contact and keep them apart in other parts of the sequence. However such reconstruction violates the gradual change prior as it enforces a drastic intrinsic change in the object's geometry at the transition between the glued and unglued sub-sequences. Instead consistent global reconstruction should keep the surfaces separate throughout the entire sequence. We introduce a new method for globally consistent space-time geometry and motion reconstruction from video capture. We use the gradual change prior to resolve inconsistencies and faithfully reconstruct the geometry and motion of the scanned objects. In contrast to most previous methods our algorithm doesn't require a strong shape prior such as a template and provides better results than other template-free approaches.

Sensor saturation in Fourier multiplexed imaging

Abstract: Optically multiplexed image acquisition techniques have become increasingly popular for encoding different exposures, color channels, light fields, and other properties of light onto two-dimensional image sensors. Recently, Fourier-based multiplexing and reconstruction approaches have been introduced in order to achieve a superior light transmission of the employed modulators and better signal-to-noise characteristics of the reconstructed data. We show in this paper that Fourier-based reconstruction approaches suffer from severe artifacts in the case of sensor saturation, i.e. when the dynamic range of the scene exceeds the capabilities of the image sensor. We analyze the problem, and propose a novel combined optical light modulation and computational reconstruction method that not only suppresses such artifacts, but also allows us to recover a wider dynamic range than existing image-space multiplexing approaches.

Optical Image Processing Using Light Modulation Displays

Abstract: We propose to enhance the capabilities of the human visual system by performing optical image processing directly on an observed scene. Unlike previous work which additively superimposes imagery on a scene, or completely replaces scene imagery with a manipulated version, we perform all manipulation through the use of a light modulation display to spatially filter incoming light. We demonstrate a number of perceptually motivated algorithms including contrast enhancement and reduction, object highlighting for preattentive emphasis, colour saturation, de-saturation and de-metamerization, as well as visual enhancement for the colour blind. A camera observing the scene guides the algorithms for on-the-fly processing, enabling dynamic application scenarios such as monocular scopes, eyeglasses and windshields.

A theory of plenoptic multiplexing

Abstract: Multiplexing is a common technique for encoding high-dimensional image data into a single, two-dimensional image. Examples of spatial multiplexing include Bayer patterns to capture color channels, and integral images to encode light fields. In the Fourier domain, optical heterodyning has been used to acquire light fields. In this paper, we develop a general theory of multiplexing the dimensions of the plenoptic function onto an image sensor. Our theory enables a principled comparison of plenoptic multiplexing schemes, including noise analysis, as well as the development of a generic reconstruction algorithm. The framework also aides in the identification and optimization of novel multiplexed imaging applications.

Binocular Camera Calibration Using Rectification Error

Abstract: Reprojection error is a commonly used measure for comparing the quality of different camera calibrations, for example when choosing the best calibration from a set. While this measure is suitable for single cameras, we show that we can improve calibrations in a binocular or multi-camera setup by calibrating the cameras in pairs using a rectification error. The rectification error determines the mismatch in epipolar constraints between a pair of cameras, and it can be used to calibrate binocular camera setups more accurately than using the reprojection error. We provide a quantitative comparison of the reprojection and rectification errors, and also demonstrate our result with examples of binocular stereo reconstruction.

Light reallocation for high contrast projection using an analog micromirror array

Abstract: We demonstrate for the first time a proof of concept projector with a secondary array of individually controllable, analog micromirrors added to improve the contrast and peak brightness of conventional projectors. The micromirrors reallocate the light of the projector lamp from the dark parts towards the light parts of the image, before it reaches the primary image modulator. Each element of the analog micromirror array can be tipped/tilted to divert portions of the light from the lamp in two dimensions. By directing these mirrors on an image-dependent basis, we can increase both the peak intensity of the projected image as well as its contrast.In this paper, we describe and analyze the optical design for projectors using this light reallocation approach. We also discuss software algorithms to compute the best light reallocation pattern for a given input image, using the constraints of real hardware. We perform extensive simulations of this process to evaluate image quality and performance characteristics of this process. Finally, we present a first proof-of-concept implementation of this approach using a prototype analog micromirror device.

On-the-fly tone mapping for backward-compatible high dynamic range image/video compression

Abstract: In this paper, we propose a real-time tone-mapping scheme for backward compatible high dynamic range (HDR) video compression. The appropriate choice of a tone-mapping operator (TMO) can significantly improve the HDR quality reconstructed from a low dynamic range (LDR) version. We develop a statistical model that approximates the mean square error (MSE) distortion resulting from the combined processes of tone-mapping and compression. Using this model, we formulate a numerical optimization problem to find the tone-curve that minimizes the expected MSE in the reconstructed HDR sequence. We then simplify the developed model in order to reduce the computational complexity of the optimization problem to a closed-form solution. Performance evaluations show that the proposed methods provide superior performance in terms of HDR MSE and SSIM compared to existing tone-mapping schemes. It is also shown that the LDR image quality resulting from the proposed methods matches that produced by perceptually-based TMOs.

Defocus Techniques For Camera Dynamic Range Expansion

Abstract: Defocus imaging techniques, involving the capture and reconstruction of purposely out-of-focus images, have recently become feasible due to advances in deconvolution methods. This paper evaluates the feasibility of defocus imaging as a means of increasing the effective dynamic range of conventional image sensors. Blurring operations spread the energy of each pixel over the surrounding neighborhood; bright regions transfer energy to nearby dark regions, reducing dynamic range. However, there is a trade-off between image quality and dynamic range inherent in all conventional sensors. The approach involves optically blurring the captured image by turning the lens out of focus, modifying that blurred image with a filter inserted into the optical path, then recovering the desired image by deconvolution. We analyze the properties of the setup to determine whether any combination can produce a dynamic range reduction with acceptable image quality. Our analysis considers both properties of the filter to measure local contrast reduction, as well as the distribution of image intensity at different scales as a measure of global contrast reduction. Our results show that while combining state-of-the-art aperture filters and deconvolution methods can reduce the dynamic range of the defocused image, providing higher image quality than previous methods, rarely does the loss in image fidelity justify the improvements in dynamic range.

Calibration of the visual difference predictor for estimating visibility of JPEG2000 compression artifacts in CT images

Abstract: Many visual difference predictors (VDPs) have used basic psychophysical data (such as ModelFest) to calibrate the algorithm parameters and to validate their performances. However, the basic psychophysical data often do not contain sufficient number of stimuli and its variations to test more complex components of a VDP. In this paper we calibrate the Visual Difference Predictor for High Dynamic Range images (HDR-VDP) using radiologists' experimental data for JPEG2000 compressed CT images which contain complex structures. Then we validate the HDR-VDP in predicting the presence of perceptible compression artifacts. 240 CT-scan images were encoded and decoded using JPEG2000 compression at four compression ratios (CRs). Five radiologists participated to independently determine if each image pair (original and compressed images) was indistinguishable or distinguishable. A threshold CR for each image, at which 50% of radiologists would detect compression artifacts, was estimated by fitting a psychometric function. The CT images compressed at the threshold CRs were used to calibrate the HDR-VDP parameters and to validate its prediction accuracy. Our results showed that the HDR-VDP calibrated for the CT image data gave much better predictions than the HDR-VDP calibrated to the basic psychophysical data (ModelFest + contrast masking data for sine gratings).

Quality-preserving image downsizing

Abstract: The image quality of a digital viewfinder is considerably lower than that of a through-the-lens optical system. While the sensor may be capable of capturing 10 or 20 megapixels, the screen of the viewfinder is typically constrained to resolutions under 1 megapixel. The limited resolution makes it impossible to discern all the small details of the captured image. Small blurs and noise that are present in the full-size image can render the image unusable for certain tasks, yet these artifacts may be too small to be discernible in the downsampled version shown on the camera viewfinder.