Showing papers by "Wolfgang Heidrich published in 2017"
21 Jul 2017
TL;DR: This work introduces a deep learning solution to video deblurring, where a CNN is trained end-to-end to learn how to accumulate information across frames, and shows that the features learned extend todeblurring motion blur that arises due to camera shake in a wide range of videos.
Abstract: Motion blur from camera shake is a major problem in videos captured by hand-held devices. Unlike single-image deblurring, video-based approaches can take advantage of the abundant information that exists across neighboring frames. As a result the best performing methods rely on the alignment of nearby frames. However, aligning images is a computationally expensive and fragile procedure, and methods that aggregate information must therefore be able to identify which regions have been accurately aligned and which have not, a task that requires high level scene understanding. In this work, we introduce a deep learning solution to video deblurring, where a CNN is trained end-to-end to learn how to accumulate information across frames. To train this network, we collected a dataset of real videos recorded with a high frame rate camera, which we use to generate synthetic motion blur for supervision. We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.
499 citations
TL;DR: A new OCT denoising algorithm is introduced that combines a novel speckle noise model, derived from local statistics of empirical spectral domain OCT (SD-OCT) data, with a Huber variant of total variation regularization for edge preservation.
Abstract: Optical coherence tomography (OCT) is a non-invasive technique with a large array of applications in clinical imaging and biological tissue visualization. However, the presence of speckle noise affects the analysis of OCT images and their diagnostic utility. In this article, we introduce a new OCT denoising algorithm. The proposed method is founded on a numerical optimization framework based on maximum-a-posteriori estimate of the noise-free OCT image. It combines a novel speckle noise model, derived from local statistics of empirical spectral domain OCT (SD-OCT) data, with a Huber variant of total variation regularization for edge preservation. The proposed approach exhibits satisfying results in terms of speckle noise reduction as well as edge preservation, at reduced computational cost.
81 citations
TL;DR: A concept for a lens attachment that turns a standard DSLR camera and lens into a light field camera that combines patch-based and depth-based synthesis in a novel fashion and achieves substantial improvements in super-resolution for side-view images as well as the high-quality and view-coherent rendering of dense and high-resolution light fields.
Abstract: We propose a concept for a lens attachment that turns a standard DSLR camera and lens into a light field camera. The attachment consists of eight low-resolution, low-quality side cameras arranged around the central high-quality SLR lens. Unlike most existing light field camera architectures, this design provides a high-quality 2D image mode, while simultaneously enabling a new high-quality light field mode with a large camera baseline but little added weight, cost, or bulk compared with the base DSLR camera. From an algorithmic point of view, the high-quality light field mode is made possible by a new light field super-resolution method that first improves the spatial resolution and image quality of the side cameras and then interpolates additional views as needed. At the heart of this process is a super-resolution method that we call iterative Patch- And Depth-based Synthesis (iPADS), which combines patch-based and depth-based synthesis in a novel fashion. Experimental results obtained for both real captured data and synthetic data confirm that our method achieves substantial improvements in super-resolution for side-view images as well as the high-quality and view-coherent rendering of dense and high-resolution light fields.
61 citations
TL;DR: This work extends an established 2D method, Particle Imaging Velocimetry, to three dimensions by encoding depth into color by illuminating the flow volume with a continuum of light planes (a "rainbow").
Abstract: Despite significant recent progress, dense, time-resolved imaging of complex, non-stationary 3D flow velocities remains an elusive goal. In this work we tackle this problem by extending an established 2D method, Particle Imaging Velocimetry, to three dimensions by encoding depth into color. The encoding is achieved by illuminating the flow volume with a continuum of light planes (a "rainbow"), such that each depth corresponds to a specific wavelength of light. A diffractive component in the camera optics ensures that all planes are in focus simultaneously. With this setup, a single color camera is sufficient for tracking 3D trajectories of particles by combining 2D spatial and 1D color information. For reconstruction, we derive an image formation model for recovering stationary 3D particle positions. 3D velocity estimation is achieved with a variant of 3D optical flow that accounts for both physical constraints as well as the rainbow image formation model. We evaluate our method with both simulations and an experimental prototype setup.
57 citations
11 Apr 2017
TL;DR: By learning CSC features from large-scale image datasets for the first time, this paper achieves significant quality improvements in a number of imaging tasks and enables new applications in high-dimensional feature learning that has been intractable using existing CSC methods.
Abstract: Convolutional sparse coding (CSC) is a promising direction for unsupervised learning in computer vision. In contrast to recent supervised methods, CSC allows for convolutional image representations to be learned that are equally useful for high-level vision tasks and low-level image reconstruction and can be applied to a wide range of tasks without problem-specific retraining. Due to their extreme memory requirements, however, existing CSC solvers have so far been limited to low-dimensional problems and datasets using a handful of low-resolution example images at a time. In this paper, we propose a new approach to solving CSC as a consensus optimization problem, which lifts these limitations. By learning CSC features from large-scale image datasets for the first time, we achieve significant quality improvements in a number of imaging tasks. Moreover, the proposed method enables new applications in high-dimensional feature learning that has been intractable using existing CSC methods. This is demonstrated for a variety of reconstruction problems across diverse problem domains, including 3D multispectral demosaicing and 4D light field view synthesis.
35 citations
TL;DR: This work derives a detailed image formation model for the setting of holographic projection displays, as well as a multiplexing method based on a combination of phase retrieval methods and complex matrix factorization.
Abstract: Computational caustics and light steering displays offer a wide range of interesting applications, ranging from art works and architectural installations to energy efficient HDR projection. In this work we expand on this concept by encoding several target images into pairs of front and rear phase-distorting surfaces. Different target holograms can be decoded by mixing and matching different front and rear surfaces under specific geometric alignments. Our approach, which we call mix-and-match holography, is made possible by moving from a refractive caustic image formation process to a diffractive, holographic one. This provides the extra bandwidth that is required to multiplex several images into pairing surfaces.We derive a detailed image formation model for the setting of holographic projection displays, as well as a multiplexing method based on a combination of phase retrieval methods and complex matrix factorization. We demonstrate several application scenarios in both simulation and physical prototypes.
21 citations
TL;DR: A new class of sensor is introduced, the Coded Wavefront Sensor, which provides high spatio-temporal resolution and phase accuracy better than 0.1 wavelengths at reconstruction rates of 50 Hz or more, thus opening up many new applications from high-resolution adaptive optics to real-time phase retrieval in microscopy.
Abstract: Wavefront sensors and more general phase retrieval methods have recently attracted a lot of attention in a host of application domains, ranging from astronomy to scientific imaging and microscopy. In this paper, we introduce a new class of sensor, the Coded Wavefront Sensor, which provides high spatio-temporal resolution using a simple masked sensor under white light illumination. Specifically, we demonstrate megapixel spatial resolution and phase accuracy better than 0.1 wavelengths at reconstruction rates of 50 Hz or more, thus opening up many new applications from high-resolution adaptive optics to real-time phase retrieval in microscopy.
20 citations
TL;DR: This paper proposes a discrim inative transfer learning method that incorporates formal proximal optimization and discriminative learning for general image restoration and allows for reuse across various problems and conditions while achieving an efficiency comparable to previous discriminatives approaches.
Abstract: Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency. However, these methods require separate training for each restoration task (e.g., denoising, deblurring, demosaicing) and problem condition (e.g., noise level of input images). This makes it time-consuming and difficult to encompass all tasks and conditions during training. In this paper, we propose a discriminative transfer learning method that incorporates formal proximal optimization and discriminative learning for general image restoration. The method requires a single-pass training and allows for reuse across various problems and conditions while achieving an efficiency comparable to previous discriminative approaches. Furthermore, after being trained, our model can be easily transferred to new likelihood terms to solve untrained tasks, or be combined with existing priors to further improve image restoration quality.
20 citations
25 Dec 2017
TL;DR: This work revisits the pixel-wise similarity between different color channels of the image and accordingly proposes a novel algorithm for correcting chromatic aberration based on this cross-channel correlation, which leads to significant quality improvements for large chromatic aberrations.
Abstract: Image aberrations can cause severe degradation in image quality for consumer-level cameras, especially under the current tendency to reduce the complexity of lens designs in order to shrink the overall size of modules. In simplified optical designs, chromatic aberration can be one of the most significant causes for degraded image quality, and it can be quite difficult to remove in post-processing, since it results in strong blurs in at least some of the color channels. In this work, we revisit the pixel-wise similarity between different color channels of the image and accordingly propose a novel algorithm for correcting chromatic aberration based on this cross-channel correlation. In contrast to recent weak prior-based models, ours uses strong pixel-wise fitting and transfer, which lead to significant quality improvements for large chromatic aberrations. Experimental results on both synthetic and real world images captured by different optical systems demonstrate that the chromatic aberration can be significantly reduced using our approach.
19 citations
01 May 2017
TL;DR: This paper first summarizes recent progress in the field of light steering projectors in cinema and then, based on new projector and existing display characteristics, proposes the inclusion of two simple display attributes: Maximum Average Luminance and Peak (Color) Primary Luminance.
Abstract: New light steering projectors in cinema form images by moving light away from dark regions into bright areas of an image In these systems, the peak luminance of small features can far exceed full screen white luminance In traditional projectors where light is filtered or blocked in order to give shades of gray (or colors), the peak luminance is fixed The luminance of chromatic features benefit in the same way as white features, and chromatic image details can be reproduced at high brightness leading to a much wider overall color gamut coverage than previously possible Projectors of this capability are desired by the creative community to aid in and enhance storytelling Furthermore, reduced light source power requirements of light steering projectors provide additional economic and environmental benefits While the dependency of peak luminance level on (bright) image feature size is new in the digital cinema space, display technologies with identical characteristics such as OLED, LED LCD and Plasma TVs are well established in the home Similarly, direct view LED walls are popular in events, advertising and architectural markets To enable consistent color reproduction across devices in today’s content production pipelines, models that describe modern projectors and display attributes need to evolve together with HDR standards and available metadata This paper is a first step towards rethinking legacy display descriptors such as contrast, peak luminance and color primaries in light of new display technology We first summarize recent progress in the field of light steering projectors in cinema and then, based on new projector and existing display characteristics propose the inclusion of two simple display attributes: Maximum Average Luminance and Peak (Color) Primary Luminance We show that the proposed attributes allow a better prediction of content reproducibility on HDR displays To validate this assertion, we test professional content on a commercial HDR television system and show that the proposed attributes better predict if a pixel value lies inside the capabilities of a display or not
2 citations