Showing papers on "High dynamic range published in 2018"
08 Sep 2018
TL;DR: This paper proposes the first non-flow-based deep framework for high dynamic range (HDR) imaging of dynamic scenes with large-scale foreground motions, and produces excellent results where color artifacts and geometric distortions are significantly reduced compared to existing state-of-the-art methods.
Abstract: This paper proposes the first non-flow-based deep framework for high dynamic range (HDR) imaging of dynamic scenes with large-scale foreground motions. In state-of-the-art deep HDR imaging, input images are first aligned using optical flows before merging, which are still error-prone due to occlusion and large motions. In stark contrast to flow-based methods, we formulate HDR imaging as an image translation problem without optical flows. Moreover, our simple translation network can automatically hallucinate plausible HDR details in the presence of total occlusion, saturation and under-exposure, which are otherwise almost impossible to recover by conventional optimization approaches. Our framework can also be extended for different reference images. We performed extensive qualitative and quantitative comparisons to show that our approach produces excellent results where color artifacts and geometric distortions are significantly reduced compared to existing state-of-the-art methods, and is robust across various inputs, including images without radiometric calibration.
220 citations
15 Jan 2018
TL;DR: In this article, the authors present a state estimation pipeline that fuses event cameras and standard frames, and inertial measurements, and demonstrate the first autonomous quadrotor flight using an event camera.
Abstract: Event cameras are bioinspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. However, event cameras output only little information when the amount of motion is limited, such as in the case of almost still motion. Conversely, standard cameras provide instant and rich information about the environment most of the time (in low-speed and good lighting scenarios), but they fail severely in case of fast motions, or difficult lighting such as high dynamic range or low light scenes. In this letter, we present the first state estimation pipeline that leverages the complementary advantages of these two sensors by fusing in a tightly coupled manner events, standard frames, and inertial measurements. We show on the publicly available Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement of 130% over event-only pipelines, and 85% over standard-frames-only visual-inertial systems, while still being computationally tractable. Furthermore, we use our pipeline to demonstrate-to the best of our knowledge-the first autonomous quadrotor flight using an event camera for state estimation, unlocking flight scenarios that were not reachable with traditional visual-inertial odometry, such as low-light environments and high dynamic range scenes. Videos of the experiments: http://rpg.ifi.uzh.ch/ultimateslam.html.
196 citations
TL;DR: This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet, which accepts LDR images as input and generates images with an expanded range in an end‐to‐end fashion.
Abstract: High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end-to-end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.
164 citations
TL;DR: This model provides useful guidelines to optimize the mini-LED backlit LCDs for achieving dynamic contrast ratio comparable to organic LED displays and suppress the halo effect to indistinguishable level.
Abstract: We analyze the performance of high dynamic range liquid crystal displays (LCDs) using a two-dimensional local dimming mini-LED backlight. The halo effect of such a HDR display system is investigated by both numerical simulation and human visual perception experiment. The halo effect is mainly governed by two factors: intrinsic LCD contrast ratio (CR) and dimming zone number. Based on our results, to suppress the halo effect to indistinguishable level, a LCD with CR≈5000:1 requires about 200 local dimming zones, while for a LCD with CR≈2000:1 the required dimming zone number is over 3000. Our model provides useful guidelines to optimize the mini-LED backlit LCDs for achieving dynamic contrast ratio comparable to organic LED displays.
142 citations
TL;DR: This paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map built via classic dense reconstruction pipelines, and tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency.
Abstract: Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that—because of the technological advantages of the event camera—our pipeline works in scenes characterized by high-speed motion, which are still inaccessible to standard cameras.
141 citations
121 citations
08 Sep 2018
TL;DR: The proposed method is the first framework to create high dynamic range images based on the estimated multi-exposure stack using the conditional generative adversarial network structure and is significantly similar to the ground truth than other state-of-the-art algorithms.
Abstract: High dynamic range images contain luminance information of the physical world and provide more realistic experience than conventional low dynamic range images. Because most images have a low dynamic range, recovering the lost dynamic range from a single low dynamic range image is still prevalent. We propose a novel method for restoring the lost dynamic range from a single low dynamic range image through a deep neural network. The proposed method is the first framework to create high dynamic range images based on the estimated multi-exposure stack using the conditional generative adversarial network structure. In this architecture, we train the network by setting an objective function that is a combination of L1 loss and generative adversarial network loss. In addition, this architecture has a simplified structure than the existing networks. In the experimental results, the proposed network generated a multi-exposure stack consisting of realistic images with varying exposure values while avoiding artifacts on public benchmarks, compared with the existing methods. In addition, both the multi-exposure stacks and high dynamic range images estimated by the proposed method are significantly similar to the ground truth than other state-of-the-art algorithms.
104 citations
TL;DR: In this article, a convolutional neural network (CNN) was used to reconstruct an HDR image from a single low dynamic range (LDR) image, and the final HDR image can be formed by merging these inference results.
Abstract: Recently, high dynamic range (HDR) imaging has attracted much attention as a technology to reflect human visual characteristics owing to the development of the display and camera technology. This paper proposes a novel deep neural network model that reconstructs an HDR image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively simple for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range but also has the advantage of restoring the light information of the actual physical world. The proposed method is an end-to-end reconstruction process, and it has the advantage of being able to easily combine a network to extend an additional range. In the experimental results, the proposed method shows quantitative and qualitative improvement in performance, compared with the conventional algorithms.
101 citations
Posted Content•
TL;DR: ExpandNet as discussed by the authors uses a CNN to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction, and uses a multiscale architecture which avoids the use of upsampling layers to improve image quality.
Abstract: High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end-to-end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.
81 citations
TL;DR: In this paper, a closed-loop frequency-locking scheme was proposed to simultaneously track Zeeman-split resonance pairs of nitrogen-vacancy (NV) centers in diamond, which offers a three-orders-of-magnitude increase in dynamic range compared to open-loop methodologies.
Abstract: We demonstrate a robust, scale-factor-free vector magnetometer, which uses a closed-loop frequency-locking scheme to simultaneously track Zeeman-split resonance pairs of nitrogen-vacancy (NV) centers in diamond. This technique offers a three-orders-of-magnitude increase in dynamic range compared to open-loop methodologies; is robust against fluctuations in temperature, resonance linewidth, and contrast; and allows for simultaneous interrogation of multiple transition frequencies. By directly detecting the resonance frequencies of NV centers oriented along each of the diamond's four tetrahedral crystallographic axes, we perform full vector reconstruction of an applied magnetic field.
78 citations
TL;DR: An optical design and a rendering pipeline for a full-color volumetric near-eye display which simultaneously presents imagery with near-accurate per-pixel focus across an extended volume, allowing the viewer to accommodate freely across this entire depth range.
Abstract: We introduce an optical design and a rendering pipeline for a full-color volumetric near-eye display which simultaneously presents imagery with near-accurate per-pixel focus across an extended volume ranging from 15cm (67 diopters) to 4M (025 diopters), allowing the viewer to accommodate freely across this entire depth range This is achieved using a focus-tunable lens that continuously sweeps a sequence of 280 synchronized binary images from a high-speed, Digital Micromirror Device (DMD) projector and a high-speed, high dynamic range (HDR) light source that illuminates the DMD images with a distinct color and brightness at each binary frame Our rendering pipeline converts 3-D scene information into a 2-D surface of color voxels, which are decomposed into 280 binary images in a voxel-oriented manner, such that 280 distinct depth positions for full-color voxels can be displayed
03 Sep 2018
TL;DR: This paper proposes an event-driven OF algorithm called adaptive block-matching optical flow (ABMOF), which uses time slices of accumulated DVS events and developed both ABMOF and Lucas-Kanade (LK) algorithms using the authors' adapted slices.
Abstract: Dynamic Vision Sensors (DVS) output asynchronous log intensity change events. They have potential applications in high-speed robotics, autonomous cars and drones. The precise event timing, sparse output, and wide dynamic range of the events are well suited for optical flow, but conventional optical flow (OF) algorithms are not well matched to the event stream data. This paper proposes an event-driven OF algorithm called adaptive block-matching optical flow (ABMOF). ABMOF uses time slices of accumulated DVS events. The time slices are adaptively rotated based on the input events and OF results. Compared with other methods such as gradient-based OF, ABMOF can efficiently be implemented in compact logic circuits. We developed both ABMOF and Lucas-Kanade (LK) algorithms using our adapted slices. Results shows that ABMOF accuracy is comparable with LK accuracy on natural scene data including sparse and dense texture, high dynamic range, and fast motion exceeding 30,000 pixels per second.
17 Jun 2018
TL;DR: This paper develops a method for recovery of $K-sparse, sum-of-sinusoids from finitely many wrapped samples, thus avoiding clipping or saturation, and obtains a parametric sampling theorem.
Abstract: In parallel to Shannon's sampling theorem, the recent theory of unlimited sampling yields that a bandlimited function with high dynamic range can be recovered exactly from oversampled, low dynamic range samples. In this way, the unlimited sampling methodology circumvents the dynamic range problem that limits the use of conventional analog-to-digital converters (ADCs) which are prone to clipping or saturation problem. The unlimited sampling theorem is made practicable by using a unique ADC architecture-the self-reset ADC or the SR-ADC-which resets voltage before clipping, thus producing modulo or wrapped samples. While retaining full dynamic range of the input signal, surprisingly, the sampling density prescribed by the unlimited sampling theorem is independent of the maximum recordable voltage of the new ADC and depends only on the signal bandwidth. As the corresponding problem of signal recovery from such modulo samples arises in various applications with different signal models, where the original result does not directly apply, the original paper continues to trigger research follow-ups. In this paper, we investigate the case of sampling and reconstruction of a mixture of $K$ sinusoids from such modulo samples. This problem is at the heart of spectral estimation theory and application areas include active sensing, ranging, source localization, interferometry and direction-of-arrival estimation. By relying on the SR-ADCs, we develop a method for recovery of $K$ -sparse, sum-of-sinusoids from finitely many wrapped samples, thus avoiding clipping or saturation. As our signal model is completely characterized by $K$ pairs of amplitudes and frequencies, we obtain a parametric sampling theorem; we complement it with a recovery algorithm. Numerical demonstrations validate the effectivity of our approach.
TL;DR: A novel method for 3D shape measurement of high-contrast surfaces in real-time by taking advantage of the transitioning state of digital micromirror device and 1-bit binary patterns to bypass the rigid camera-projector synchronization requirement is introduced.
20 Oct 2018
TL;DR: In this article, a high-dynamic-range polarization imaging sensor inspired by the visual system of the mantis shrimp was presented, which achieves 140dB dynamic range and 61dB maximum signal-to-noise ratio across 384×288 pixels equipped with logarithmic photodiodes.
Abstract: Polarization is one of the three fundamental properties of light, along with color and intensity, yet most vertebrate species, including humans, are blind with respect to this light modality. In contrast, many invertebrates, including insects, spiders, cephalopods, and stomatopods, have evolved to detect polarization information with high-dynamic-range photosensitive cells and utilize this information in visually guided behavior. In this paper, we present a high-dynamic-range polarization imaging sensor inspired by the visual system of the mantis shrimp. Our bioinspired imager achieves 140 dB dynamic range and 61 dB maximum signal-to-noise ratio across 384×288 pixels equipped with logarithmic photodiodes. Contrary to state-of-the-art active pixel sensors, where photodiodes in individual pixels operate in reverse bias mode and yield up to ∼60 dB dynamic range, our pixel has a logarithmic response by operating individual photodiodes in forward bias mode. This novel pixel circuitry is monolithically integrated with pixelated polarization filters composed of 250-nm-tall × 75-nm-wide aluminum nanowires to enable snapshot polarization imaging at 30 frames per second. This sensor can enable many automotive and remote sensing applications, where high-dynamic-range imaging augmented with polarization information can provide critical information during hazy or rainy conditions.
TL;DR: In this article, the structure of high-density SiPMs and their performance is presented, including photon detection efficiency, the photon number resolving capabilities, the linearity versus photon flux, and the single-photon time resolution.
Abstract: Silicon photomultipliers (SiPM) are photodetectors that have obtained a growing attention in the last years. They have single photon sensitivity, but also a high dynamic range: the output current signal is proportional to the number of detected photons and it is possible to distinguish up to tens of photon per each light pulse. In Fondazione Bruno Kessler (Trento, Italy) we developed the so called high-density SiPMs, with narrow trenches to isolate the cells (i.e. single-photon avalanche diodes) and with high fill factor. These detectors feature high photon detection efficiency and high dynamic range. Moreover, the reduction of the cell size reduces the correlated noise (i.e. lower crosstalk between cells and lower afterpulsing probability) and it makes the single-cell response faster, increasing the maximum counting rate. These characteristics can be very important in applications like light detection and ranging, spectroscopy, and physics experiments. In this paper, we present the structure of high-density SiPMs and their performance. In particular, the photon detection efficiency, the photon number resolving capabilities, the linearity versus photon flux, and the single-photon time resolution, investigating how these parameters change with the cell pitch.
08 Sep 2018
TL;DR: In this article, a temporal propagation network (TPN) is proposed to propagate a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation mask, where the properties are available for only a few key-frames.
Abstract: Videos contain highly redundant information between frames. Such redundancy has been studied extensively in video compression and encoding, but is less explored for more advanced video processing. In this paper, we propose a learnable unified framework for propagating a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation mask, where the properties are available for only a few key-frames. Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner. We theoretically prove two essential properties of TPN: (a) by regularizing the global transformation matrix as orthogonal, the “style energy” of the property can be well preserved during propagation; and (b) such regularization can be achieved by the proposed switchable TPN with bi-directional training on pairs of frames. We apply the switchable TPN to three tasks: colorizing a gray-scale video based on a few colored key-frames, generating an HDR video from a low dynamic range (LDR) video and a few HDR frames, and propagating a segmentation mask from the first frame in videos. Experimental results show that our approach is significantly more accurate and efficient than the state-of-the-art methods.
TL;DR: Experimental results demonstrate that the proposed technique can accurately measure objects with an HDR of surface reflectivity variation, and can enhance the dynamic range of the fringe projection profilometry system.
Abstract: Fringe projection profilometry has been widely used in many fields for its advantages such as high speed, high accuracy, and robustness to environmental illumination and surface texture However, it is vulnerable to high dynamic range (HDR) objects To this end, we propose a technique that can enhance the dynamic range of the fringe projection profilometry system According to the surface reflectivities of the measured objects, several groups of fringe patterns with optimal light intensities are generated based on the intensity response function of a camera The HDR fringe images are acquired by fusing these fringe patterns, and a three-step phase-shifting algorithm is used to obtain the unwrapped phase from the fused images Experimental results demonstrate that the proposed technique can accurately measure objects with an HDR of surface reflectivity variation
TL;DR: This paper examines methods to best exploit the High Dynamic Range of the single photon avalanche diode (SPAD) in a high fill-factor HDR photon counting pixel that is scalable to megapixel arrays and combines multi-exposure HDR with temporal oversampling in-pixel.
Abstract: This paper examines methods to best exploit the High Dynamic Range (HDR) of the single photon avalanche diode (SPAD) in a high fill-factor HDR photon counting pixel that is scalable to megapixel arrays. The proposed method combines multi-exposure HDR with temporal oversampling in-pixel. We present a silicon demonstration IC with 96 × 40 array of 8.25 µm pitch 66% fill-factor SPAD-based pixels achieving >100 dB dynamic range with 3 back-to-back exposures (short, mid, long). Each pixel sums 15 bit-planes or binary field images internally to constitute one frame providing 3.75× data compression, hence the 1k frames per second (FPS) output off-chip represents 45,000 individual field images per second on chip. Two future projections of this work are described: scaling SPAD-based image sensors to HDR 1 MPixel formats and shrinking the pixel pitch to 1-3 µm.
TL;DR: It is mathematically proved that once a pixel’s modulation is larger than a threshold, the phase quality of this pixel can be considered satisfactory and this threshold can be used to guide the calculation of the needed exposure times.
01 May 2018
TL;DR: A novel database of HDR and SDR images captured in different conditions, including various capturing motions, scenes and devices is presented, concluding that capturing conditions and devices themselves can have an impact on source identification.
Abstract: Digital source identification is one of the most important problems in the field of multimedia forensics. While Standard Dynamic Range (SDR) images are commonly analyzed, High Dynamic Range (HDR) images are a less common research subject, which leaves space for further analysis. In this paper, we present a novel database of HDR and SDR images captured in different conditions, including various capturing motions, scenes and devices. As a possible application of this dataset, the performance of the well-known reference pattern noise-based source identification algorithm was tested on both kinds of images. Results have shown difficulties in source identification conducted on HDR images, due to their complexity and wider dynamic range. It is concluded that capturing conditions and devices themselves can have an impact on source identification, thus leaving space for more research in this field.
TL;DR: Experimental results show that the proposed two-layer iTMO can recover the HDR accurately so that it is possible to use these local TMOs in scalable HDR image coding schemes.
Abstract: Tone mapping operators (TMOs) and inverse TMOs (iTMOs) are important for scalable coding of high dynamic range (HDR) images. Because of the high nonlinearity of local TMOs, it is very difficult to estimate the iTMO accurately for a local TMO. In this letter, we present a two-layer local iTMO estimation algorithm using an edge-preserving decomposition technique. The low dynamic range (LDR) image is first linearized and then decomposed into a base layer and a detail layer via a fast edge-preserving decomposition method. The base layer of the HDR image is generated by subtracting the LDR detail layer from the HDR image. An iTMO function is finally estimated by solving a novel quadratic optimization problem formulated on the pair of base layers rather than the pair of HDR and LDR images as in existing methods. Experimental results show that the proposed two-layer iTMO can recover the HDR accurately so that it is possible to use these local TMOs in scalable HDR image coding schemes.
TL;DR: An algorithm for high dynamic range (HDR) and super-resolution (SR) imaging from a single image based on the Retinex approach and a recent single image SR method based on a convolutional neural network (CNN).
Abstract: This paper presents an algorithm for high dynamic range (HDR) and super-resolution (SR) imaging from a single image. First, we propose a new single image HDR imaging (HDRI) method based on the Retinex approach and exploit a recent single image SR method based on a convolutional neural network (CNN). Among many possible configurations of HDR and SR, we find an optimal system configuration and color manipulation strategy from the extensive experiments. Specifically, the best results are obtained when we first process the luminance component ( $Y$ ) of input with our single image HDRI algorithm and then feed the enhanced HDR luminance to the CNN-based SR architecture that is trained by only luminance component. The ranges of chromatic components ( $U$ and $V$ ) are just scaled in proportion to the enhanced HDR luminance, and then they are bicubic interpolated or fed to the above CNN-based SR. Subjective and objective assessments for various experiments are presented to validate the effectiveness of the proposed HDR/SR imaging scheme.
Patent•
25 Dec 2018TL;DR: In this article, a first camera is configured with exposure settings that are optimized for brighter regions, while a second camera assembly is optimized for darker regions, where the image data from both sets have values within a noise floor and a saturation level.
Abstract: High dynamic range images are generated using conventional dynamic range cameras. A first camera is configured with exposure settings that are optimized for brighter regions, while a second camera assembly is optimized for darker regions. The cameras can be rectified and can capture concurrently such that objects are relatively aligned, with global and local misregistrations being minimized. The image data is analyzed to determine regions where the image data from one camera or the other provides higher quality, such as where the brightness values fall between a noise floor and a saturation level. If image data from both sets have values within that range then the values can be combined, such as with a weighted average. A composite image is generated that includes more uniform color and brightness than in either image individually, or that could have been captured using a single camera of similar cost and capabilities.
TL;DR: An objective evaluation shows that PTF exhibits improved quality at a range of bit-rates and, due to its straightforward nature, is highly suited for real-time HDR video applications.
Abstract: High dynamic range (HDR) imaging enables the full range of light in a scene to be captured, transmitted and displayed. However, uncompressed 32-bit HDR is four times larger than traditional low dynamic range (LDR) imagery. If HDR is to fulfil its potential for use in live broadcasts and interactive remote gaming, fast, efficient compression is necessary for HDR video to be manageable on existing communications infrastructure. A number of methods have been put forward for HDR video compression. However, these can be relatively complex and frequently require the use of multiple video streams. In this paper, we propose the use of a straightforward Power Transfer Function (PTF) as a practical, computationally fast, HDR video compression solution. The use of PTF is presented and evaluated against four other HDR video compression methods. An objective evaluation shows that PTF exhibits improved quality at a range of bit-rates and, due to its straightforward nature, is highly suited for real-time HDR video applications.
TL;DR: In this article, a massively multiplexed single-photon detector is presented, which exhibits a dynamic range of 123 dB, from optical energies as low as 10 − 7 − 2.5
Abstract: Detecting light is fundamental to all optical experiments and applications. At the single photon level, the quantised nature of light requires specialised detectors, which typically saturate for more than one photon, rendering the measurement of bright light impossible. Saturation can be partially overcome by multiplexing single-photon-sensitive detectors, enabling measurement up to tens of photons. However, current approaches are still far from bridging the gap to bright light levels. Here, we report on a massively-multiplexed single-photon detector, which exhibits a dynamic range of 123 dB, from optical energies as low as $\mathbf{10^{-7}}$ photons per pulse to $\mathbf{\sim2.5\times10^{5}}$ photons per pulse. This allows us to calibrate a single photon detector directly to a power meter. The use of a single-photon sensitive detector further allows us to characterise the nonclassical features of a variety of quantum states. This device will find application where high dynamic range and single-photon sensitivity are required.
TL;DR: A single exposure dynamic rage (SEHDR) signal is obtained by introducing a triple-gain pixel and a low noise dual-gain readout circuit, which inherently mitigates the artifacts from moving objects or time-varying light sources that can appear in the multiple exposure high dynamic range (MEHDR).
Abstract: To respond to the high demand for high dynamic range imaging suitable for moving objects with few artifacts, we have developed a single-exposure dynamic range image sensor by introducing a triple-gain pixel and a low noise dual-gain readout circuit. The developed 3 μm pixel is capable of having three conversion gains. Introducing a new split-pinned photodiode structure, linear full well reaches 40 ke-. Readout noise under the highest pixel gain condition is 1 e- with a low noise readout circuit. Merging two signals, one with high pixel gain and high analog gain, and the other with low pixel gain and low analog gain, a single exposure dynamic rage (SEHDR) signal is obtained. Using this technology, a 1/2.7", 2M-pixel CMOS image sensor has been developed and characterized. The image sensor also employs an on-chip linearization function, yielding a 16-bit linear signal at 60 fps, and an intra-scene dynamic range of higher than 90 dB was successfully demonstrated. This SEHDR approach inherently mitigates the artifacts from moving objects or time-varying light sources that can appear in the multiple exposure high dynamic range (MEHDR) approach.
Posted Content•
TL;DR: This paper proposes a learnable unified framework for propagating a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation information, where the properties are available for only a few key-frames.
Abstract: Videos contain highly redundant information between frames. Such redundancy has been extensively studied in video compression and encoding, but is less explored for more advanced video processing. In this paper, we propose a learnable unified framework for propagating a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation information, where the properties are available for only a few key-frames. Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner. We theoretically prove two essential factors for TPN: (a) by regularizing the global transformation matrix as orthogonal, the "style energy" of the property can be well preserved during propagation; (b) such regularization can be achieved by the proposed switchable TPN with bi-directional training on pairs of frames. We apply the switchable TPN to three tasks: colorizing a gray-scale video based on a few color key-frames, generating an HDR video from a low dynamic range (LDR) video and a few HDR frames, and propagating a segmentation mask from the first frame in videos. Experimental results show that our approach is significantly more accurate and efficient than the state-of-the-art methods.
TL;DR: A new asymmetric sigmoid curve (ASC) based on the model of retinal adaptation encompassing symmetrical S-shaped curve is introduced, and two global tone mapping operators are presented by using the ASC by showing a high performance to state-of-the-art globaltone mapping operators.
Abstract: Global tone mapping operators using the symmetrical retinal response model to light tend to produce a low dynamic range (LDR) image that exhibits loss of details of its corresponding high dynamic range (HDR) image in a bright or dark area. In this paper, we introduce a new asymmetric sigmoid curve (ASC) based on the model of retinal adaptation encompassing symmetrical S-shaped curve, and present two global tone mapping operators by using the ASC. In the proposed method, an ASC-based tone mapping function is obtained by using a well-known classic photography technique, called zone system. In addition, a contrast-enhancing tone mapping function is introduced by formulating a bi-criteria optimization problem with the luminance histogram of an input HDR image and the ASC-based mapping function. Experimental results demonstrate that the proposed method enhances the global contrast while preserving image details in the tone-mapped LDR image. Moreover, the objective assessment results using an image quality metric indicate that the proposed method shows a high performance to state-of-the-art global tone mapping operators.