Showing papers in "IEEE Transactions on Computational Imaging in 2018"
TL;DR: A gradient ascent-based algorithm, which starts from any initial point in the space of all possible images and iteratively moves towards the direction that improves MEF-SSIM until convergence, and the final high quality fused image appears to have little dependence on the initial image.
Abstract: We propose a multi-exposure image fusion (MEF) algorithm by optimizing a novel objective quality measure, namely the color MEF structural similarity (MEF-SSIM $_c$ ) index. The design philosophy we introduce here is substantially different from existing ones. Instead of pre-defining a systematic computational structure for MEF ( e.g. , multiresolution transformation and transform domain fusion followed by image reconstruction), we directly operate in the space of all images, searching for the image that optimizes MEF-SSIM $_c$ . Specifically, we first construct the MEF-SSIM $_c$ index by improving upon and expanding the application scope of the existing MEF-SSIM algorithm. We then describe a gradient ascent-based algorithm, which starts from any initial point in the space of all possible images and iteratively moves towards the direction that improves MEF-SSIM $_c$ until convergence. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality fused images both visually and in terms of MEF-SSIM $_c$ . The final high quality fused image appears to have little dependence on the initial image. The proposed optimization framework is readily extensible to construct better MEF algorithms when better objective quality models for MEF are available.
151 citations
TL;DR: A thorough set of performance comparisons indicates a very wide range of performance differences among the existing and proposed methods, and clearly identifies those that are the most effective.
Abstract: Convolutional sparse representations are a form of sparse representation with a dictionary that has a structure that is equivalent to convolution with a set of linear filters. While effective algorithms have recently been developed for the convolutional sparse coding problem, the corresponding dictionary learning problem is substantially more challenging. Furthermore, although a number of different approaches have been proposed, the absence of thorough comparisons between them makes it difficult to determine which of them represents the current state of the art. The present work both addresses this deficiency and proposes some new approaches that outperform existing ones in certain contexts. A thorough set of performance comparisons indicates a very wide range of performance differences among the existing and proposed methods, and clearly identifies those that are the most effective.
150 citations
TL;DR: Performance achieved on simulated images as well as on a real optical setup is superior to the state-of-the-art monocular depth estimation methods (both with respect to the depth accuracy and required processing power), and is competitive with more complex and expensivedepth estimation methods such as light-field cameras.
Abstract: Depth estimation from a single image is a well-known challenge in computer vision. With the advent of deep learning, several approaches for monocular depth estimation have been proposed, all of which have inherent limitations due to the scarce depth cues that exist in a single image. Moreover, these methods are very demanding computationally, which makes them inadequate for systems with limited processing power. In this paper, a phase-coded aperture camera for depth estimation is proposed. The camera is equipped with an optical phase mask that provides unambiguous depth-related color characteristics for the captured image. These are used for estimating the scene depth map using a fully convolutional neural network. The phase-coded aperture structure is learned jointly with the network weights using backpropagation. The strong depth cues (encoded in the image by the phase mask, designed together with the network weights) allow a much simpler neural network architecture for faster and more accurate depth estimation. Performance achieved on simulated images as well as on a real optical setup is superior to the state-of-the-art monocular depth estimation methods (both with respect to the depth accuracy and required processing power), and is competitive with more complex and expensive depth estimation methods such as light-field cameras.
89 citations
TL;DR: It is shown that it is possible to obtain an explicit formula for computing the gradient of an iterative forward model with respect to the unknown object, thus enabling fast image reconstruction with the state-of-the-art fast iterative shrinkage/thresholding algorithm.
Abstract: Multiple scattering of an electromagnetic wave as it passes through an object is a fundamental problem that limits the performance of current imaging systems. In this paper, we describe a new technique—called Series Expansion with Accelerated Gradient Descent on the Lippmann–Schwinger Equation—for robust imaging under multiple scattering based on a combination of an iterative forward model and a total variation regularizer. The proposed method can account for multiple scattering, which makes it advantageous in applications where single scattering approximations are inaccurate. Specifically, the method relies on a series expansion of the scattered wave with an accelerated-gradient method. This expansion guarantees the convergence of the forward model even for strongly scattering objects. One of our key insights is that it is possible to obtain an explicit formula for computing the gradient of an iterative forward model with respect to the unknown object, thus enabling fast image reconstruction with the state-of-the-art fast iterative shrinkage/thresholding algorithm. The proposed method is validated on diffraction tomography, where complex electric field is captured at different illumination angles.
78 citations
TL;DR: The proposed Discrete Shearlet Transform Transform (DST) as a new embedding domain for blind image watermarking shows greater windowing flexibility with more sensitive to directional and anisotropic features when compared against discrete wavelet and contourlets.
Abstract: Blind watermarking targets the challenging recovery of the watermark when the host is not available during the detection stage. This paper proposes Discrete Shearlet Transform (DST) as a new embedding domain for blind image watermarking. Our novel DST blind watermark detection system uses a nonadditive scheme based on the statistical decision theory. It first computes the Probability Density Function (PDF) of the DST coefficients modeled as a Laplacian distribution. The resulting likelihood ratio is compared with a decision threshold calculated using Neyman–Pearson criterion to minimize the missed detection subject to a fixed false alarm probability. Our method is evaluated in terms of imperceptibility, robustness, and payload against different attacks (Gaussian noise, blurring, cropping, compression, and rotation) using 30 standard grayscale images covering different characteristics (smooth, more complex with a lot of edges, and high detail textured regions). The proposed method shows greater windowing flexibility with more sensitive to directional and anisotropic features when compared against discrete wavelet and contourlets.
70 citations
TL;DR: In this paper, the authors demonstrate that the presence of natural occluders in the hidden scene can obviate the need for collecting time-resolved measurements, and develop an accompanying analysis for such systems and their generalizations.
Abstract: Active non-line-of-sight imaging systems are of growing interest for diverse applications. The most commonly proposed approaches to date rely on exploiting time-resolved measurements, i.e., measuring the time it takes for short-duration light pulses to transit the scene. This typically requires expensive, specialized, ultrafast lasers, and detectors that must be carefully calibrated. We develop an alternative approach that exploits the valuable role that natural occluders in a scene play in enabling accurate and practical image formation in such settings without such hardware complexity. In particular, we demonstrate that the presence of occluders in the hidden scene can obviate the need for collecting time-resolved measurements, and develop an accompanying analysis for such systems and their generalizations. Ultimately, the results suggest the potential to develop increasingly sophisticated future systems that are able to identify and exploit diverse structural features of the environment to reconstruct scenes hidden from view.
59 citations
TL;DR: In this paper, a deep neural network is proposed to map block-wise compressive measurements of the scene to the desired image blocks in real-time, and the reconstruction of an image becomes a simple forward pass through the network and can be done in real time.
Abstract: Traditional algorithms for compressive sensing recovery are computationally expensive and are ineffective at low measurement rates. In this paper, we propose a data-driven noniterative algorithm to overcome the shortcomings of earlier iterative algorithms. Our solution, ReconNet , is a deep neural network, which is learned end-to-end to map block-wise compressive measurements of the scene to the desired image blocks. Reconstruction of an image becomes a simple forward pass through the network and can be done in real time. We show empirically that our algorithm yields reconstructions with higher peak signal-to-noise ratios (PSNRs) compared to iterative algorithms at low measurement rates and in presence of measurement noise. We also propose a variant of ReconNet, which uses adversarial loss in order to further improve reconstruction quality. We discuss how adding a fully connected layer to the existing ReconNet architecture allows for jointly learning the measurement matrix and the reconstruction algorithm in a single network. Experiments on real data obtained from a block compressive imager show that our networks are robust to unseen sensor noise.
58 citations
TL;DR: It is shown that OCAI is robust to code amplitude and code phase imbalance introduced by imperfect transmitter (TX) and receiver (RX) hardware, while also mitigating common impairments of low cost direct-conversion receivers, such as RX selfjamming and DC offsets.
Abstract: Emerging metasurface antenna technology enables flexible and low cost massive multiple-input multiple-output (MIMO) millimeter-wave (mmW) imaging for applications such as personnel screening, weapon detection, reconnaissance, and remote sensing. This work proposes an orthogonal coded active illumination (OCAI) approach which utilizes simultaneous, mutually orthogonal coded transmit signals to illuminate the scene being imaged. It is shown that OCAI is robust to code amplitude and code phase imbalance introduced by imperfect transmitter (TX) and receiver (RX) hardware, while also mitigating common impairments of low cost direct-conversion receivers, such as RX selfjamming and DC offsets. The coding gain offered by this approach improves imager signal to noise ratio performance by up to 15 dB using codes of symbol length 32. We present validation images of resolution targets and a human-scale mannequin, obtained with a custom massive-MIMO mmW imager having 24 simultaneous TXs and 72 simultaneous RXs operating in the K-band (17.5 GHz to 26.5 GHz). The imager leverages both spatial coding via frequency diverse metasurface antennas, and temporal coding via OCAI of the scene.
49 citations
TL;DR: A parametric level set (PLS) based reconstruction scheme for an EIT-imaging of conductivity distributions within multiphase systems to solve the inverse problem of finding interfaces between the regions having different conductivity values and estimating the conductivities of phases.
Abstract: Electrical impedance tomography (EIT) is an imaging modality that provides cross-sectional images of objects that carry contrasts in electrical conductivity. EIT is suitable for example for monitoring industrial processes involving multiple phases with different conductivities. This paper presents a parametric level set (PLS) based reconstruction scheme for an EIT-imaging of conductivity distributions within multiphase systems. The proposed scheme involves applying a multiphase level set model to solve the inverse problem of finding interfaces between the regions having different conductivity values and estimating the conductivities of phases. The unknown conductivity to be reconstructed is assumed to be piecewise constant while the interface between the regions are represented by the two PLS functions employing Gaussian radial basis functions (GRBF). The level set-based scheme handles topological merging and breaking naturally during the evolution process. It also offers several advantages compared to the traditional pixel-based approaches. For example, the representation of the PLS function by using GRBF provides flexibility in describing a large class of shapes with fewer unknowns. Numerical simulations and phantom experiments are performed to validate the proposed method.
43 citations
TL;DR: Experimental results show that the proposed method outperforms state-of-the-art spectral reflectance recovery methods in terms of both objective metrics and subjective visual quality.
Abstract: The spectral reflectance is an intrinsic and discriminative characteristic of object materials and can be obtained by hyperspectral imaging. However, existing hyperspectral cameras are limited in low-spatial/temporal resolution while yet being complicated and expensive. In this paper, we present a nonnegative sparse representation based method to recover high-quality spectral reflectance from a single RGB image. Unlike previous methods, our approach learns multiple nonnegative sparse coding dictionaries from the training spectral dataset in terms of clustering results. Then, the spectral reflectance of the input RGB image is recovered based on nonnegative sparse representation, which also considers the spatial structured similarity and high correlation across spectra under the learned dictionaries. Furthermore, the illumination spectrum can be estimated with the recovered spectral reflectance under the known RGB camera spectral sensitivity. Experimental results show that the proposed method outperforms state-of-the-art spectral reflectance recovery methods in terms of both objective metrics and subjective visual quality. Besides, we show an application of our method to accurately relight scenes under the novel illumination.
43 citations
TL;DR: A new way to model VLBI measurements that allows for the recovery of both the appearance and dynamics of an evolving source by reconstructing a video rather than a static image is presented.
Abstract: Very long baseline interferometry (VLBI) makes it possible to recover the images of astronomical sources with extremely high angular resolution. Most recently, the Event Horizon Telescope (EHT) has extended VLBI to short millimeter wavelengths with a goal of achieving angular resolution sufficient for imaging the event horizons of nearby supermassive black holes. Interferometry provides measurements related to the underlying source image through a sparse set spatial frequencies. An image can then be recovered from these measurements by making assumptions about the underlying image. One of the most important assumptions made by conventional imaging methods is that over the course of a night's observation the image is static. However, for quickly evolving sources, such as the galactic center's supermassive black hole (SgrA*) targeted by the EHT, this assumption is violated and these conventional imaging approaches fail. This paper presents a new way to model VLBI measurements that allows for the recovery of both the appearance and dynamics of an evolving source by reconstructing a video rather than a static image. By modeling VLBI measurements using a Gaussian Markov Model, information can be propagated across observations in time to reconstruct a video, while simultaneously learning about the dynamics of the source's emission region. This paper demonstrates our proposed expectation-maximization algorithm, StarWarps, on realistic synthetic observations of black holes, and shows how it substantially improves the results compared to conventional imaging methods. In addition to synthetic data, the technique is demonstrated on real VLBA data of the M87 jet.
TL;DR: A new family of X-ray source scanning trajectories for large-angle cone-beam computed tomography that provides greater data acquisition efficiency, and reduced reconstruction artifacts when compared with helical trajectory, and also possesses an effective preconditioner for fast iterative tomographic reconstruction.
Abstract: We present a new family of X-ray source scanning trajectories for large-angle cone-beam computed tomography. Traditional scanning trajectories are described by continuous paths through space, e.g., circles, saddles, or helices, with a large degree of redundant information in adjacent projection images. Here, we consider discrete trajectories as a set of points that uniformly sample the entire space of possible source positions, i.e., a space-filling trajectory (SFT). We numerically demonstrate the advantageous properties of the SFT when compared with circular and helical trajectories as follows: first, the most isotropic sampling of the data, second, optimal level of mutually independent data, and third, an improved condition number of the tomographic inverse problem. The practical implications of these properties in tomography are also illustrated by simulation. We show that the SFT provides greater data acquisition efficiency, and reduced reconstruction artifacts when compared with helical trajectory. It also possesses an effective preconditioner for fast iterative tomographic reconstruction.
TL;DR: A new algorithm is presented to image ground moving targets and estimate their motion parameters in a synthetic aperture radar system based on improved axis rotation-time reversal transform (IAR-TRT), which has a relatively low computational complexity.
Abstract: In this paper, a new algorithm is presented to image ground moving targets and estimate their motion parameters in a synthetic aperture radar system based on improved axis rotation-time reversal transform (IAR-TRT). In this algorithm, the second-order Keystone transform) is applied to correct the range curvature, where the Doppler ambiguity caused by a fast-moving target is considered. Then, residual linear range migration and Doppler frequency migration are eliminated by IAR-TRT, which performs an improved axis rotation transform to correct the range walk and subsequently realizes the coherent integration based on time reversal transform. Finally, a ground moving target is well focused. Additionally, the proposed method has a relatively low computational complexity since the exhaustive searching for Doppler chirp rate estimation is avoided. The effectiveness of the proposed algorithm is validated by both simulated and real data.
TL;DR: A two-dimensional travel time tomography method, which regularizes the inversion by modeling groups of slowness pixels from discrete slowned maps, called patches, as sparse linear combinations of atoms from a dictionary, is developed.
Abstract: We develop a two-dimensional travel time tomography method, which regularizes the inversion by modeling groups of slowness pixels from discrete slowness maps, called patches, as sparse linear combinations of atoms from a dictionary. We propose to use dictionary learning during the inversion to adapt dictionaries to specific slowness maps. This patch regularization, called the local model, is integrated into the overall slowness map, called the global model. The local model considers small-scale variations using a sparsity constraint, and the global model considers larger-scale features constrained using $\ell _2$ regularization. This strategy in a locally sparse travel time tomography (LST) approach enables simultaneous modeling of smooth and discontinuous slowness features. This is in contrast to conventional tomography methods, which constrain models to be exclusively smooth or discontinuous. We develop a maximum a posteriori formulation for LST and exploit the sparsity of slowness patches using dictionary learning. The LST approach compares favorably with smoothness and total variation regularization methods on densely, but irregularly sampled synthetic slowness maps.
TL;DR: In this article, a supervised learning approach for dynamic sparse sampling (SLADS) is proposed, where the objective is to find the pixel that maximizes the expected reduction in distortion (ERD) given previous measurements.
Abstract: Sparse sampling schemes can broadly be classified into two main categories: static sampling where the sampling pattern is predetermined, and dynamic sampling where each new measurement location is selected based on information obtained from previous measurements. Dynamic sampling methods are particularly appropriate for pointwise imaging methods, in which pixels are measured sequentially in arbitrary order. Examples of pointwise imaging schemes include certain implementations of atomic force microscopy, electron back scatter diffraction, and synchrotron X-ray imaging. In these pointwise imaging applications, dynamic sparse sampling methods have the potential to dramatically reduce the number of measurements required to achieve a desired level of fidelity. However, the existing dynamic sampling methods tend to be computationally expensive and are, therefore, too slow for many practical applications. In this paper, we present a framework for dynamic sampling based on machine learning techniques, which we call a supervised learning approach for dynamic sampling (SLADS). In each step of SLADS, the objective is to find the pixel that maximizes the expected reduction in distortion (ERD) given previous measurements. SLADS is fast because we use a simple regression function to compute the ERD, and it is accurate because the regression function is trained using datasets that are representative of the specific application. In addition, we introduce an approximate method to terminate dynamic sampling at a desired level of distortion. We then extend our algorithm to incorporate multiple measurements at each step, which we call groupwise SLADS. Finally, we present results on computationally generated synthetic data and experimentally collected data to demonstrate a dramatic improvement over state-of-the-art static sampling methods
TL;DR: In this article, an optimal threshold design framework was proposed to improve the signal-to-noise ratio (SNO) of the reconstructed image by using a set of oracle results to theoretically inform the maximally achievable performance.
Abstract: Quanta image sensor is a binary imaging device envisioned to be the next generation image sensor after CCD and CMOS. Equipped with a massive number of single photon detectors, the sensor has a threshold $q$ above which the number of arriving photons will trigger a binary response “1”, or “0” otherwise. Existing methods in the device literature typically assume that $q = 1$ uniformly. We argue that a spatial-temporally varying threshold can significantly improve the signal-to-noise ratio of the reconstructed image. In this paper, we present an optimal threshold design framework. We make two contributions. First, we derive a set of oracle results to theoretically inform the maximally achievable performance. We show that the oracle threshold should match exactly with the underlying pixel intensity. Second, we show that around the oracle threshold there exists a set of thresholds that give asymptotically unbiased reconstructions. The asymptotic unbiasedness has a phase transition behavior which allows us to develop a practical threshold update scheme using a bisection method. Experimentally, the new threshold design method achieves better rate of convergence than existing methods.
TL;DR: To solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained.
Abstract: Light fields suffer from a fundamental resolution tradeoff between the angular and the spatial domain. In this paper, we present a novel cross-scale light field super-resolution approach (up to $\text{8}\times$ resolution gap) to super-resolve low-resolution (LR) light field images that are arranged around a high-resolution (HR) reference image. To bridge the enormous resolution gap between the cross-scale inputs, we introduce an intermediate view denoted as single image super-resolution (SISR) image, i.e., super-resolving LR input via single image based super-resolution scheme, which owns identical resolution as HR image yet lacks high-frequency details that SISR scheme cannot recover under such significant resolution gap. By treating the intermediate SISR image as the low-frequency part of our desired HR image, the remaining issue of recovering high-frequency components can be effectively solved by the proposed high-frequency compensation super-resolution (HCSR) method. Essentially, HCSR works by transferring as much as possible the high-frequency details from the HR reference view to the LR light field image views. Moreover, to solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, we compute multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained. The superiority of our proposed HCSR method is validated on extensive datasets including synthetic, real-world and challenging microscope scenes.
TL;DR: It is demonstrated that, under physically realistic conditions, the covariance of the data has a super-resolution power corresponding to the squared magnitude of the imager point spread function.
Abstract: Speckle-based imaging consists of forming a super-resolved reconstruction of an unknown sample from low-resolution images obtained under random inhomogeneous illuminations (speckles). In a blind context, where the illuminations are unknown, we study the intrinsic capacity of speckle-based imagers to recover spatial frequencies outside the frequency support of the data, with minimal assumptions about the sample. We demonstrate that, under physically realistic conditions, the covariance of the data has a super-resolution power corresponding to the squared magnitude of the imager point spread function. This theoretical result is important for many practical imaging systems such as acoustic and electromagnetic tomographs, fluorescence and photoacoustic microscopes, or synthetic aperture radar imaging. A numerical validation is presented in the case of fluorescence microscopy.
TL;DR: A new unmixing model for multitemporal hyperspectral images accounting for smooth temporal variations, construed as spectral variability, and abrupt spectral changes interpreted as outliers is proposed.
Abstract: Hyperspectral unmixing is a blind source separation problem that consists in estimating the reference spectral signatures contained in a hyperspectral image, as well as their relative contribution to each pixel according to a given mixture model In practice, the process is further complexified by the inherent spectral variability of the observed scene and the possible presence of outliers More specifically, multitemporal hyperspectral images, ie, sequences of hyperspectral images acquired over the same area at different time instants, are likely to simultaneously exhibit moderate endmember variability and abrupt spectral changes either due to outliers or to significant time intervals between consecutive acquisitions Unless properly accounted for, these two perturbations can significantly affect the unmixing process In this context, we propose a new unmixing model for multitemporal hyperspectral images accounting for smooth temporal variations, construed as spectral variability, and abrupt spectral changes interpreted as outliers The proposed hierarchical Bayesian model is inferred using a Markov chain Monte Carlo method allowing the posterior of interest to be sampled and Bayesian estimators to be approximated A comparison with unmixing techniques from the literature on synthetic and real data allows the interest of the proposed approach to be appreciated
TL;DR: This paper proposes to seek positive patterns that are linear combinations of the desired patterns (with negative values), and the linear transformation matrices are chosen to reject the dark current, and shows that pattern generalization can be solved using a semi nonnegative matrix factorization algorithm.
Abstract: A single-pixel camera is a computational imaging device that only requires a single point detector to capture the image of a scene. This device measures the inner product of the scene and the spatial light modulator patterns. The image of the scene can be recovered through postprocessing the measurements obtained for a set of different patterns. Independent of the strategy used for image recovery, real acquisitions require the spatial light modulator patterns to be positive. In addition, the dark current measured in the absence of modulation must be rejected. To date, both experimental issues have been addressed empirically. In this paper, we solve these from a general perspective. Indeed, we propose to seek positive patterns that are linear combinations of the desired patterns (with negative values), and the linear transformation matrices are chosen to reject the dark current. We refer to the problem of finding the positive patterns and the linear combinations as “pattern generalization.” To the best of our knowledge, this is the first time that this problem has been introduced. In addition, we show that pattern generalization can be solved using a semi nonnegative matrix factorization algorithm. The data obtained from simulations demonstrate that our approach performs similarly to or better than conventional methods, while using fewer measurements.
TL;DR: In this article, a hierarchical Bayesian model is proposed to solve the problem of deconvolution and restoration of optical endomicroscopy (OEM) data and three estimation algorithms are compared to exploit the resulting joint posterior distribution.
Abstract: Optical endomicroscopy (OEM) is an emerging technology platform with preclinical and clinical imaging applications. Pulmonary OEM via fibre bundles has the potential to provide in vivo, in situ molecular signatures of disease such as infection and inflammation. However, enhancing the quality of data acquired by this technique for better visualization and subsequent analysis remains a challenging problem. Cross coupling between fiber cores and sparse sampling by imaging fiber bundles are the main reasons for image degradation, and poor detection performance (i.e., inflammation, bacteria, etc.). In this paper, we address the problem of deconvolution and restoration of OEM data. We propose a hierarchical Bayesian model to solve this problem and compare three estimation algorithms to exploit the resulting joint posterior distribution. The first method is based on Markov chain Monte Carlo methods, however, it exhibits a relatively long computational time. The second and third algorithms deal with this issue and are based on a variational Bayes approach and an alternating direction method of multipliers algorithm, respectively. Results on both synthetic and real datasets illustrate the effectiveness of the proposed methods for restoration of OEM images.
TL;DR: This paper constructs a unitary transform for modeling deep holographic scenes using a generalization of linear canonical transforms and derives necessary invertibility conditions for the diffraction from nonplanar surfaces for symmetric light propagation kernels, such as Fresnel diffraction.
Abstract: With the advent of ultrahigh-resolution holographic displays, viewing macroscopic deep scenes with large viewing angles becomes a possibility. These deep holograms possess different signal properties in contrast with common applications where the scene content is assumed to lie around a planar slice. Therefore, the conventional approach of refocusing at a fixed depth is ineffective. There is a need for an efficient invertible transformation that is able to account for the wide depth range of macroscopic three-dimensional scenes. To this end, we derive necessary invertibility conditions for the diffraction from nonplanar surfaces for symmetric light propagation kernels, such as Fresnel diffraction. We construct a unitary transform for modeling deep holographic scenes using a generalization of linear canonical transforms. From the symplectic properties of the time–frequency domain, we obtain invertibility conditions of the transforms depending on surface shape, hologram bandwidth, and wavelength. These transforms can be subsequently combined with other sparsifying transforms for compression. Experiments demonstrate one application in lossy coding of holograms by implementing a computationally efficient subset of the transforms for piecewise depth profiles that is combined with the JPEG 2000 codec. Results show improved reconstruction quality. A significant visual gain is observed as the depth information is well preserved under identical encoding rates in contrast to using Fresnel propagation at a fixed depth. This paper shows that it is possible to effectively represent holograms of variable-depth scenes and our local adaptive transform leads to a practical holographic compression framework.
TL;DR: This paper examines the use of re-projection alignment (RA) to estimate per-radiograph geometry and finds that for either trajectory, slowly varying vertical errors cannot be reliably estimated by employing the RA method alone; such errors are indistinguishable from a trajectory of different pitch.
Abstract: For standard laboratory microtomography systems, acquired radiographs do not always adhere to the strict geometrical assumptions of the reconstruction algorithm. The consequence of this geometrical inconsistency is that the reconstructed tomogram contains motion artifacts, e.g., blurring, streaking, double-edges. To achieve a motion-artifact-free tomographic reconstruction, one must estimate, and subsequently correct for, the per-radiograph experimental geometry parameters. In this paper, we examine the use of re-projection alignment (RA) to estimate per-radiograph geometry. Our simulations evaluate how the convergence properties of RA vary with: motion-type (smooth versus random), trajectory (helical versus discrete-sampling ‘space-filling’ trajectories) and tomogram resolution. The idealized simulations demonstrate for the space-filling trajectory that RA convergence rate and accuracy is invariant with regard to the motion-type and that the per-projection motions can be estimated to less than 0.25 pixel mean absolute error by performing a single quarter-resolution RA iteration followed by a single half-resolution RA iteration. The direct impact is that, for the space-filling trajectory, one can incorporate RA in an iterative multi-grid reconstruction scheme with only a single RA iteration per multi-grid resolution step. We also find that for either trajectory, slowly varying vertical errors cannot be reliably estimated by employing the RA method alone; such errors are indistinguishable from a trajectory of different pitch. This has minimal effect in practice because RA can be combined with reference frame correction which is effective for correcting low-frequency errors.
TL;DR: A greedy algorithm named clustered maximal projection on minimal eigenspace (CMPME) is proposed to select the most informative sampling positions based on some optimality criteria to get the (near-) optimal spatial sampling scheme for 3-D arrays.
Abstract: In this paper, sampling design of three-dimensional (3-D) synthetic array (i.e., synthetic volume array) for microwave imaging is considered. Generally, the spatial sampling criteria for one- or two-dimensional arrays can be determined based on some narrowband/ultrawideband array theories. However, for 3-D arrays, where antennas are located in a volume instead of over a surface, these existing array theories are no longer straightforwardly applicable. To address the spatial sampling problem of 3-D arrays, we formulate it as a sensor/observation selection problem in this paper. Although some selection approaches exist and are conveniently applicable to small-scale problems, they are either less efficient or provide less optimal results for selection problems with data dimensions of hundreds or even thousands which is typical for microwave imaging. To get the (near-) optimal spatial sampling scheme for 3-D arrays, a greedy algorithm named clustered maximal projection on minimal eigenspace (CMPME) is proposed to select the most informative sampling positions based on some optimality criteria. This algorithm attempts to select the fewest sampling positions by considering an error threshold for the estimated images. Moreover, it has higher computational efficiency compared to the existing approaches. Finally, its effectiveness and selection performances are demonstrated through some imaging examples.
TL;DR: A synthetic phantom simulation illustrates that the proposed algorithm enables the aforementioned CT system to achieve high quality images by minimizing artifacts induced by limited-angle data and beam hardening.
Abstract: We propose a new iterative X-ray computed tomography (CT) reconstruction algorithm for electron beam X-ray tomography of multiphase flows in metal pipes. This application uses limited-angle projections due to the fixed configuration, and semiconductor-type energy-integrating detectors. For the data-fitting objective function, the proposed method incorporates a nonlinear Gaussian model with object-dependent variance to approximate the compound Poisson distribution, and a dual material decomposition based on images of the volume fractions of metal (titanium) and liquid (water). The volume fraction-based material decomposition enables us to use a maximum sum constraint that helps address the ill-posed nature of the problem. Two different regularizers, $\ell _0$ norm and edge-preserving hyperbola regularizers, are applied differently on each volume fraction image based on the characteristics of objects in each image. A synthetic phantom simulation illustrates that the proposed algorithm enables the aforementioned CT system to achieve high quality images by minimizing artifacts induced by limited-angle data and beam hardening.
TL;DR: These novel modalities can be used for applications requiring high signal-to-noise ratio, long-range, low-power, and low payload such as microsatellites and uninhabited aerial vehicles; passive imaging using sources of opportunity; and in applications requiring spectrum efficiency.
Abstract: This paper introduces two novel imaging modalities: Doppler displaced phase center antenna (Doppler-DPCA) and Doppler along track interferometry (Doppler-ATI). The DPCA and ATI techniques have the distinct advantage of removing the response from stationary targets (clutter). We develop DPCA and ATI techniques in Doppler synthetic aperture radar (Doppler-SAR) paradigm to image moving targets embedded in clutter. Doppler-SAR uses ultra-narrowband continuous waveforms (UNCW) to reconstruct high-resolution SAR images. We consider a two-channel bistatic configuration with a stationary antenna transmitting UNCW and two receiving antennas moving along an arbitrary trajectory in tandem. We introduced the theory for Doppler-DPCA and Doppler-ATI. We derive an interferometric phase model and develop equations of velocity mapping. While conventional wideband SAR DPCA and ATI use range difference, Doppler-DPCA and Doppler-ATI use high-resolution temporal Doppler difference in imaging of moving targets. We present numerical results to demonstrate our theory. These novel modalities can be used for applications requiring high signal-to-noise ratio, long-range, low-power, and low payload such as microsatellites and uninhabited aerial vehicles; passive imaging using sources of opportunity, such as TV and radio stations; and in applications requiring spectrum efficiency.
TL;DR: For a given number of measurements, extensive experiments show that the proposed measurement matrix considerably increases the overall recovery performance, or equivalently decreases the number of sampled pixels for a specific recovery quality compared to random sampling matrix and Gaussian linear combinations employed by the state-of-the-art compressive sensing methods.
Abstract: This paper presents an adaptive and intelligent sparse model for digital image sampling and recovery. In the proposed sampler, we adaptively determine the number of required samples for retrieving image based on space-frequency-gradient information content of image patches. By leveraging texture in space, sparsity locations in DCT domain, and directional decomposition of gradients, the sampler structure consists of a combination of uniform, random, and nonuniform sampling strategies. For reconstruction, we model the recovery problem as a two-state cellular automaton to iteratively restore image with scalable windows from generation to generation. We demonstrate the recovery algorithm quickly converges after a few generations for an image with arbitrary degree of texture. For a given number of measurements, extensive experiments on standard image-sets, infra-red, and mega-pixel range imaging devices show that the proposed measurement matrix considerably increases the overall recovery performance, or equivalently decreases the number of sampled pixels for a specific recovery quality compared to random sampling matrix and Gaussian linear combinations employed by the state-of-the-art compressive sensing methods. In practice, the proposed measurement-adaptive sampling/recovery framework includes various applications from intelligent compressive imaging-based acquisition devices to computer vision and graphics, and image processing technology. Simulation codes are available online for reproduction purposes.
TL;DR: In this paper, the authors proposed two reconstruction algorithms for multiband images acquired by transmission electron microscopes (STEM-EELS) which exploit the spectral structure and the spatial smoothness of the image.
Abstract: Electron microscopy has shown to be a very powerful tool to map the chemical nature of samples at various scales down to atomic resolution. However, many samples can not be analyzed with an acceptable signal-to-noise ratio because of the radiation damage induced by the electron beam. This is particularly crucial for electron energy loss spectroscopy (EELS), which acquires spectral-spatial data and requires high beam intensity. Since scanning transmission electron microscopes (STEM) are able to acquire data cubes by scanning the electron probe over the sample and recording a spectrum for each spatial position, it is possible to design the scan pattern and to sample only specific pixels. As a consequence, partial acquisition schemes are now conceivable, provided a reconstruction of the full data cube is conducted as a postprocessing step. This paper proposes two reconstruction algorithms for multiband images acquired by STEM-EELS which exploits the spectral structure and the spatial smoothness of the image. The performance of the proposed schemes is illustrated thanks to experiments conducted on a realistic phantom dataset as well as real EELS spectrum-images.
TL;DR: This paper introduces a reflection removal method using single light field (LF) captures that outperforms state-of-the-art ones by at least 2.9 dB in terms of background restoration PSNR, with significantly fewer visual artifacts.
Abstract: Photos taken through reflective surfaces such as display windows inevitably suffer from visual degradations caused by reflection interferences. The superimposition of two layers, i.e., background and reflection, obscures important target features, and causes challenges for subsequent computer vision applications. In this paper, we introduce a reflection removal method using single light field (LF) captures. Through focus manipulation and subsequent optimization, our algorithm is able to separate reflection from background effectively by exploiting the subtle pixel parallax of LF data. Since it requires only a single capture of the scene, it can handle moving scenes that change quickly, as well as static ones. Extensive experiments have been carried out over both synthetic and real LF data, which show our method outperforms state-of-the-art ones by at least 2.9 dB in terms of background restoration PSNR, with significantly fewer visual artifacts.
TL;DR: In this article, the authors propose an alternate formulation where the position error of each antenna is mapped to a spatial shift operator in the image-domain, and the radar autofocus problem becomes a multichannel blind deconvolution problem, in which the radar measurements correspond to observations of a static radar image that is convolved with the spatial shift kernel associated with each antenna.
Abstract: A common problem that arises in radar imaging systems, especially those mounted on mobile platforms, is antenna position ambiguity. Approaches to resolve this ambiguity and correct position errors are generally known as radar autofocus. Common techniques that attempt to resolve the antenna ambiguity generally assume an unknown gain and phase error afflicting the radar measurements. However, ensuring identifiability and tractability of the unknown error imposes strict restrictions on the allowable antenna perturbations. Furthermore, these techniques are often not applicable in near-field imaging, where mapping the position ambiguity to phase errors breaks down. In this paper, we propose an alternate formulation where the position error of each antenna is mapped to a spatial shift operator in the image-domain. Thus, the radar autofocus problem becomes a multichannel blind deconvolution problem, in which the radar measurements correspond to observations of a static radar image that is convolved with the spatial shift kernel associated with each antenna. To solve the reformulated problem, we also develop a block coordinate descent framework that leverages the sparsity and piecewise smoothness of the radar scene, as well as the one-sparse property of the two-dimensional shift kernels. We evaluate the performance of our approach using both simulated and experimental radar measurements and demonstrate its superior performance compared with state-of-the-art methods.