Showing papers in "IEEE Transactions on Computational Imaging in 2016"

Video Super-Resolution With Convolutional Neural Networks

Abstract: Convolutional neural networks (CNN) are a special type of deep neural networks (DNN). They have so far been successfully applied to image super-resolution (SR) as well as other image restoration tasks. In this paper, we consider the problem of video super-resolution. We propose a CNN that is trained on both the spatial and the temporal dimensions of videos to enhance their spatial resolution. Consecutive frames are motion compensated and used as input to a CNN that provides super-resolved video frames as output. We investigate different options of combining the video frames within one CNN architecture. While large image databases are available to train deep neural networks, it is more challenging to create a large video database of sufficient quality to train neural nets for video restoration. We show that by using images to pretrain our model, a relatively small video database is sufficient for the training of our model to achieve and even improve upon the current state-of-the-art. We compare our proposed approach to current video as well as image SR algorithms.

Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Abstract: Many material and biological samples in scientific imaging are characterized by nonlocal repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a two-dimensional image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the nonlocal redundancy in images. We adapt a framework, termed plug-and-play priors, to solve these imaging problems in a regularized inversion setting. The power of the plug-and-play approach is that it allows a wide array of modern denoising algorithms to be used as a “prior model” for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the plug-and-play approach, and we use these insights to design a new nonlocal means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.

A General Framework for Compressed Sensing and Parallel MRI Using Annihilating Filter Based Low-Rank Hankel Matrix

Abstract: Parallel MRI (pMRI) and compressed sensing MRI (CS-MRI) have been considered as two distinct reconstruction problems. Inspired by recent k-space interpolation methods, an annihilating filter-based low-rank Hankel matrix approach is proposed as a general framework for sparsity-driven k-space interpolation method which unifies pMRI and CS-MRI. Specifically, our framework is based on a novel observation that the transform domain sparsity in the primary space implies the low-rankness of weighted Hankel matrix in the reciprocal space. This converts pMRI and CS-MRI to a k-space interpolation problem using a structured matrix completion. Experimental results using in vivo data for single/multicoil imaging as well as dynamic imaging confirmed that the proposed method outperforms the state-of-the-art pMRI and CS-MRI.

Optical Tomographic Image Reconstruction Based on Beam Propagation and Sparse Regularization

Abstract: Optical tomographic imaging requires an accurate forward model as well as regularization to mitigate missing-data artifacts and to suppress noise. Nonlinear forward models can provide more accurate interpretation of the measured data than their linear counterparts, but they generally result in computationally prohibitive reconstruction algorithms. Although sparsity-driven regularizers significantly improve the quality of reconstructed image, they further increase the computational burden of imaging. In this paper, we present a novel iterative imaging method for optical tomography that combines a nonlinear forward model based on the beam propagation method (BPM) with an edge-preserving three-dimensional (3-D) total variation (TV) regularizer. The central element of our approach is a time-reversal scheme, which allows for an efficient computation of the derivative of the transmitted wave-field with respect to the distribution of the refractive index. This time-reversal scheme together with our stochastic proximal-gradient algorithm makes it possible to optimize under a nonlinear forward model in a computationally tractable way, thus enabling a high-quality imaging of the refractive index throughout the object. We demonstrate the effectiveness of our method through several experiments on simulated and experimentally measured data.

Robust Multiframe Super-Resolution Employing Iteratively Re-Weighted Minimization

Abstract: Multiframe super-resolution algorithms reconstruct high-resolution images by exploiting complementary information in multiple low-resolution frames. However, despite their success under ideal conditions, most existing methods rely on simplistic approximations to the physics of image acquisition and show limited robustness in real-world applications. This paper proposes spatially adaptive Bayesian modeling and an iterative algorithm for robust super-resolution imaging. In particular, we introduce a weighted Gaussian observation model to consider space variant noise and weighted bilateral total variation to exploit sparsity of natural images. Based on this model, we develop a majorization–minimization algorithm implemented as iteratively re-weighted minimization. The proposed method simultaneously estimates model parameters and the super-resolved image in an iterative coarse-to-fine scheme. Compared to prior work, our approach combines the benefits of achieving robust and edge preserving image reconstruction with small amount of parameter tuning, while being flexible in terms of motion models, computationally efficient and easy to implement. Our experimental evaluation confirms that our approach outperforms state-of-the-art algorithms under various practical conditions, e.g., inaccurate geometric and photometric registration or invalid measurements.

Spectral CT Reconstruction With Image Sparsity and Spectral Mean

Abstract: Photon-counting detectors can acquire x-ray intensity data in different energy bins. The signal-to-noise ratio of resultant raw data in each energy bin is generally low due to the narrow bin width and quantum noise. To address this problem, here we propose an image reconstruction approach for spectral CT to simultaneously reconstruct x-ray attenuation coefficients in all the energy bins. Because the measured spectral data are highly correlated among the x-ray energy bins, the intraimage sparsity and interimage similarity are important prior knowledge for image reconstruction. Inspired by this observation, the total variation and spectral mean measures are combined to improve the quality of reconstructed images. For this purpose, a linear mapping function is used to minimalize image differences between energy bins. The split Bregman technique is applied to perform image reconstruction. Our numerical and experimental results show that the proposed algorithms outperform competing iterative algorithms in this context.

An Augmented Lagrangian Method for Complex-Valued Compressed SAR Imaging

Abstract: In this paper, we present a solution to the complex synthetic aperture radar (SAR) imaging problem within a constrained optimization formulation where the objective function includes a combination of the $\ell _1$ -norm and the total variation of the magnitude of the complex valued reflectivity field. The technique we present relies on recent advances in the solution of optimization problems, based on Augmented Lagrangian Methods, and in particular on the Alternating Direction Method of Multipliers (ADMM). We rigorously derive the proximal mapping operators, associated with a linear transform of the magnitude of the reflectivity vector and magnitude-total-variation cost functions, for complex-valued SAR images, and thus enable the use of ADMM techniques to obtain computationally efficient solutions for radar imaging. We study the proposed techniques with multiple features (sparse and piecewise-constant in magnitude) based on a weighted sum of the 1-norm and magnitude-total-variation. We derive a fast implementation of the algorithm using only two transforms per iteration for problems admitting unitary transforms as forward models. Experimental results on real data from TerraSAR-X and SARPER —airborne SAR system developed by ASELSAN—demonstrate the effectiveness of the proposed approach.

A Comparative Error Analysis of Current Time-of-Flight Sensors

Abstract: Time-of-flight (ToF) cameras suffer from systematic errors, which can be an issue in many application scenarios. In this paper, we investigate the error characteristics of eight different ToF cameras. Our survey covers both well established and recent cameras including the Microsoft Kinect V2. We present up to six experiments for each camera to quantify different types of errors. For each experiment, we outline the basic setup, present comparable data for each camera, and discuss the respective results. The results discussed in this paper enable the community to make appropriate decisions in choosing the best matching camera for a certain application. This work also lays the foundation for a framework to benchmark future ToF cameras. Furthermore, our results demonstrate the necessity for correcting characteristic measurement errors. We believe that the presented findings will allow 1) the development of novel correction methods for specific errors and 2) the development of general data processing algorithms that are able to robustly operate on a wider range of cameras and scenes.

Toward Long-Distance Subdiffraction Imaging Using Coherent Camera Arrays

Abstract: In this work, we propose using camera arrays coupled with coherent illumination as an effective method of improving spatial resolution in long distance images by a factor of ten and beyond. Recent advances in ptychography have demonstrated that one can image beyond the diffraction limit of the objective lens in a microscope. We demonstrate a similar imaging system to image beyond the diffraction limit in long range imaging. We emulate a camera array with a single camera attached to an $XY$ translation stage. We show that an appropriate phase retrieval based reconstruction algorithm can be used to effectively recover the lost high resolution details from the multiple low resolution acquired images. We analyze the effects of noise, required degree of image overlap, and the effect of increasing synthetic aperture size on the reconstructed image quality. We show that coherent camera arrays have the potential to greatly improve imaging performance. Our simulations show resolution gains of $10\times$ and more are achievable. Furthermore, experimental results from our proof-of-concept systems show resolution gains of $4\times - 7\times$ for real scenes. All experimental data and code is made publicly available on the project webpage. Finally, we introduce and analyze in simulation a new strategy to capture macroscopic Fourier Ptychography images in a single snapshot, albeit using a camera array.

Data-Driven Learning of a Union of Sparsifying Transforms Model for Blind Compressed Sensing

Abstract: Compressed sensing is a powerful tool in applications such as magnetic resonance imaging (MRI). It enables accurate recovery of images from highly undersampled measurements by exploiting the sparsity of the images or image patches in a transform domain or dictionary. In this work, we focus on blind compressed sensing (BCS), where the underlying sparse signal model is a priori unknown, and propose a framework to simultaneously reconstruct the underlying image as well as the unknown model from highly undersampled measurements. Specifically, our model is that the patches of the underlying image(s) are approximately sparse in a transform domain. We also extend this model to a union of transforms model that better captures the diversity of features in natural images. The proposed block coordinate descent type algorithms for BCS are highly efficient, and are guaranteed to converge to at least the partial global and partial local minimizers of the highly nonconvex BCS problems. Our numerical experiments show that the proposed framework usually leads to better quality of image reconstructions in MRI compared to several recent image reconstruction methods. Importantly, the learning of a union of sparsifying transforms leads to better image reconstructions than a single adaptive transform.

Sensitivity Encoding for Aligned Multishot Magnetic Resonance Reconstruction

Abstract: This paper introduces a framework for the reconstruction of magnetic resonance images in the presence of rigid motion. The rationale behind our proposal is to make use of the partial $k$ -space information provided by multiple receiver coils in order to estimate the position of the imaged object throughout the shots that contribute to the image. The estimated motion is incorporated into the reconstruction model in an iterative manner to obtain a motion-free image. The method is parameter-free, does not assume any prior model for the image to be reconstructed, avoids blurred images due to resampling, does not make use of external sensors, and does not require modifications in the acquisition sequence. Validation is performed using synthetically corrupted data to study the limits for full motion-recovered reconstruction in terms of the amount of motion, encoding trajectories, number of shots and availability of prior information, and to compare with the state of the art. Quantitative and visual results of its application to a highly challenging volumetric brain imaging cohort of $207$ neonates are also presented, showing the ability of the proposed reconstruction to generally improve the quality of reconstructed images, as evaluated by both sparsity and gradient entropy based metrics.

Fast Steerable Principal Component Analysis

Abstract: Cryo-electron microscopy nowadays often requires the analysis of hundreds of thousands of 2-D images as large as a few hundred pixels in each direction. Here, we introduce an algorithm that efficiently and accurately performs principal component analysis (PCA) for a large set of 2-D images, and, for each image, the set of its uniform rotations in the plane and their reflections. For a dataset consisting of $n$ images of size $L \times L$ pixels, the computational complexity of our algorithm is $O(nL^3 + L^4)$ , while existing algorithms take $O(nL^4)$ . The new algorithm computes the expansion coefficients of the images in a Fourier–Bessel basis efficiently using the nonuniform fast Fourier transform. We compare the accuracy and efficiency of the new algorithm with traditional PCA and existing algorithms for steerable PCA.

Parameter Estimation of Ground Moving Targets Based on SKT-DLVT Processing

Abstract: It is well known that the motion of a ground moving target may induce the range cell migration, spectrum spread, and velocity ambiguity during the imaging time, which makes the image smeared. To eliminate the influence of these factors on image focusing, a novel parameter estimation algorithm for ground moving targets, namely SKT-DLVT, is proposed in this paper. In this method, the segmented keystone transform (SKT) is used to correct the range walks of targets simultaneously, and a new transform, namely, Doppler Lv’s transform (DLVT) is applied on the azimuth signal to estimate the parameters, i.e., the along- and cross-track velocities of targets. Theoretical analysis confirms that no interpolation is needed for the proposed method and targets can be well focused within limited searching range of the ambiguity number. The proposed method is capable of obtaining accurate parameter estimates efficiently in low signal-to-noise ratio (SNR) scenario with low computational burden and memory cost. Thus, it is suitable to be applied in memory-limited and real-time processing systems. The effectiveness of the proposed method is demonstrated by both simulated and real data.

Two-dimensional Autofocus for Spotlight SAR Polar Format Imagery

Abstract: Conventional two-dimensional (2-D) autofocus algorithms blindly estimate the phase error in the sense that they do not exploit any a priori information on the structure of the 2-D phase error. As such, they often suffer from the low computational efficiency and the lack of data redundancy to accurately estimate the 2-D phase error. In this paper, an efficient semi-blind 2-D autofocus algorithm which is based on exploiting the a priori knowledge about the 2-D phase error structure, is presented. First, as a prerequisite of the proposed method, the analytical structure of the residual 2-D phase error in SAR imagery is investigated in the polar format algorithm framework. Then, by incorporating this a priori information, a novel 2-D autofocus approach is proposed. The new method only requires an estimate of the azimuth phase error and/or residual range cell migration, while the 2-D phase error can then be computed directly from the estimated azimuth phase error or residual range cell migration. Experimental results clearly demonstrate the effectiveness and the robustness of the proposed method.

Nonlinear Optimization Algorithm for Partially Coherent Phase Retrieval and Source Recovery

Abstract: We propose a new algorithm for recovering both complex field (phase and amplitude) and source distribution (illumination spatial coherence) from a stack of intensity images captured through focus. The joint recovery is formulated as a nonlinear least-square-error optimization problem, which is solved iteratively by a modified Gauss–Newton method. We derive the gradient and Hessian of the cost function and show that our second-order optimization approach outperforms previously proposed phase retrieval algorithms, for datasets taken with both coherent and partially coherent illumination. The method is validated experimentally in a commercial microscope with both Kohler illumination and a programmable light-emitting diode dome.

Robust Bayesian Target Detection Algorithm for Depth Imaging From Sparse Single-Photon Data

Abstract: This paper presents a new Bayesian model and associated algorithm for depth and intensity profiling using full waveforms from time-correlated single-photon counting measurements in the limit of very low photon counts (i.e., typically less than 20 photons per pixel). The model represents each Lidar waveform as an unknown constant background level, which is combined in the presence of a target, to a known impulse response weighted by the target intensity and finally corrupted by Poisson noise. The joint target detection and depth imaging problem is expressed as a pixelwise model selection and estimation problem, which is solved using Bayesian inference. Prior knowledge about the problem is embedded in a hierarchical model that describes the dependence structure between the model parameters while accounting for their constraints. In particular, Markov random fields (MRFs) are used to model the joint distribution of the background levels and of the target presence labels, which are both expected to exhibit significant spatial correlations. An adaptive Markov chain Monte Carlo algorithm including reversible-jump updates is then proposed to compute the Bayesian estimates of interest. This algorithm is equipped with a stochastic optimization adaptation mechanism that automatically adjusts the parameters of the MRFs by maximum marginal likelihood estimation. Finally, the benefits of the proposed methodology are demonstrated through a series of experiments using real data.

Joint Reconstruction of Absorbed Optical Energy Density and Sound Speed Distributions in Photoacoustic Computed Tomography: A Numerical Investigation

Abstract: Photoacoustic computed tomography (PACT) is a rapidly emerging bioimaging modality that seeks to reconstruct an estimate of the absorbed optical energy density within an object. Conventional PACT image reconstruction methods assume a constant speed-of-sound (SOS), which can result in image artifacts when acoustic aberrations are significant. It has been demonstrated that incorporating knowledge of an object’s SOS distribution into a PACT image reconstruction method can improve image quality. However, in many cases, the SOS distribution cannot be accurately and/or conveniently estimated prior to the PACT experiment. Because variations in the SOS distribution induce aberrations in the measured photoacoustic wavefields, certain information regarding an object’s SOS distribution is encoded in the PACT measurement data. Based on this observation, a joint reconstruction (JR) problem has been proposed in which the SOS distribution is concurrently estimated along with the sought-after absorbed optical energy density from the photoacoustic measurement data. A broad understanding of the extent to which the JR problem can be accurately and reliably solved has not been reported. In this work, a series of numerical experiments is described that elucidate some important properties of the JR problem that pertain to its practical feasibility. To accomplish this, an optimization-based formulation of the JR problem is developed that yields a nonlinear iterative algorithm that alternatively updates the two image estimates. Heuristic analytic insights into the reconstruction problem are also provided. These results confirm the ill-conditioned nature of the joint reconstruction problem that will present significant challenges for practical applications.

Models of Monocular and Binocular Visual Perception in Quality Assessment of Stereoscopic Images

Abstract: Assessing quality of experience (QoE) for three-dimensional (3-D) video is challenging. In this paper, we propose a new full-reference stereoscopic image quality metric, by simulating the behaviors of visual perception with simple and complex receptive field properties and constructing the models of monocular and binocular visual perception. To be more specific, the stereoscopic images are first classified into noncorresponding and corresponding regions. Then, monocular energy responses are generated for the noncorresponding region based on stimuli from different spatial frequencies and orientations, and binocular energy responses are generated for the corresponding region based on stimuli from different spatial frequencies, orientations and disparities, respectively. Finally, gradient similarities between the energy responses of the original and distorted stereoscopic images are measured for noncorresponding and corresponding regions, respectively, and all results are fused to get an overall score. Experiments on three 3-D image quality assessment (IQA) databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves higher agreement with subjective assessment, making it better suited for the evaluation and optimization of stereoscopic image processing algorithms.

Fast Multilayer Laplacian Enhancement

Abstract: A novel, fast, and practical way of enhancing images is introduced in this paper. Our approach builds on Laplacian operators of well-known edge-aware kernels, such as bilateral and nonlocal means, and extends these filter's capabilities to perform more effective and fast image smoothing, sharpening, and tone manipulation. We propose an approximation of the Laplacian, which does not require normalization of the kernel weights. Multiple Laplacians of the affinity weights endow our method with progressive detail decomposition of the input image from fine to coarse scale. These image components are blended by a structure mask, which avoids noise/artifact magnification or detail loss in the output image. Contributions of the proposed method to existing image editing tools are: 1) low computational and memory requirements, making it appropriate for mobile device implementations (e.g., as a finish step in a camera pipeline); and 2) a range of filtering applications from detail enhancement to denoising with only a few control parameters, enabling the user to apply a combination of various (and even opposite) filtering effects.

Locally Similar Sparsity-Based Hyperspectral Compressive Sensing Using Unmixing

Abstract: Linear unmixing-based compressive sensing has been extensively exploited for hyperspectral image (HSI) compression in recent years among which gradient sparsity is widely used to characterize the spatial continuity of abundance matrix given a small amount of endmembers. Though these methods have achieved good reconstruction results, identifying necessary endmembers from an HSI is challenging for them. In this study, instead of using a small amount of given endmembers, a locally similar sparsity-based hyperspectral unmixing compressive sensing (LSSHUCS) method is proposed to unmix the HSI with an established redundant endmember library. Considering that each pixel is a mixture of several endmembers, a novel locally similar sparsity constraint is imposed on the abundance matrix, which depicts the sparsity of abundance vectors and the local similarity among those sparse vectors simultaneously. This constraint guarantees to reconstruct the HSI precisely even with a quite low sample rate and can select the necessary endmembers from the endmember library automatically for unmixing. LSSHUCS is further extended to a more general one, which tactfully settles the spectrum variation problem, and an augmented Lagrangian algorithm is elaborated meticulously to solve the inverse linear problem in LSSHUCS. Extensive experimental results on both synthetic and real hyperspectral data demonstrate that the proposed method surpasses several state-of-the-art methods on reconstruction accuracy.

Color-Line Regularization for Color Artifact Removal

Abstract: Existing regularization functions for color image restoration cannot remove color artifact, undesirable appearance of colors, sufficiently, which degrades the quality of color images. In this paper, we propose a new regularization function, named the local color nuclear norm (LCNN), for removing color artifact in color image restoration. LCNN is designed to promote the local color-line property: the color distribution of each local region of a color image exhibits strong linearity. The local color-line property is inherent in clean color images and is violated by color artifact, so that suppressing LCNN is expected to reduce color artifact effectively. In addition, the very nature of LCNN allows us to incorporate it into various color image restoration formulations, where the associated optimization problems can be efficiently solved by proximal splitting techniques. Several illustrative applications of LCNN are presented with comprehensive experimental results.

Numerical Inversion of a Broken Ray Transform Arising in Single Scattering Optical Tomography

Abstract: The article presents an efficient image reconstruction algorithm for single scattering optical tomography (SSOT) in circular geometry of data acquisition. This novel medical imaging modality uses photons of light that scatter once in the body to recover its interior features. The mathematical model of SSOT is based on the broken ray (or V-line Radon) transform (BRT), which puts into correspondence to an image function its integrals along V-shaped piecewise linear trajectories. The process of image reconstruction in SSOT requires inversion of that transform. We implement numerical inversion of a broken ray transform in a disc with partial radial data. Our method is based on a relation between the Fourier coefficients of the image function and those of its BRT recently discovered by Ambartsoumian and Moon. The numerical algorithm requires solution of ill-conditioned matrix problems, which is accomplished using a half-rank truncated singular value decomposition method. Several numerical computations validating the inversion formula are presented, which demonstrate the accuracy, speed, and robustness of our method in the case of both noise-free and noisy data.

A Gaussian Mixture MRF for Model-Based Iterative Reconstruction With Applications to Low-Dose X-Ray CT

Abstract: Markov random fields (MRFs) have been widely used as prior models in various inverse problems such as tomographic reconstruction. While MRFs provide a simple and often effective way to model the spatial dependencies in images, they suffer from the fact that parameter estimation is difficult. In practice, this means that MRFs typically have very simple structure that cannot completely capture the subtle characteristics of complex images. In this paper, we present a novel Gaussian mixture Markov random field model (GM-MRF) that can be used as a very expressive prior model for inverse problems such as denoising and reconstruction. The GM-MRF forms a global image model by merging together individual Gaussian-mixture models (GMMs) for image patches. In addition, we present a novel analytical framework for computing MAP estimates using the GM-MRF prior model through the construction of surrogate functions that result in a sequence of quadratic optimizations. We also introduce a simple but effective method to adjust the GM-MRF so as to control the sharpness in low- and high-contrast regions of the reconstruction separately. We demonstrate the value of the model with experiments including image denoising and low-dose CT reconstruction.

Vehicle Tracking in Video Using Fractional Feedback Kalman Filter

Abstract: Object tracking is a challenging and important area of research. The object tracking system must be capable of tracking abrupt variations in object state. Kalman filter is fundamental and widely used as an optimal state estimator in object tracking. With known noise and system parameters, Kalman filter tends to stabilize the gain. However, during sudden transitions of the object tracked, constant gain Kalman filter may diverge. This paper proposes a modified steady-state gain of the Kalman filter, which is achieved by introducing a fractional feedback loop across the Kalman gain. The modified Kalman gain is estimated by minimizing the cost function of the proposed Kalman filter. Results show that the accuracy and robustness of the Kalman filter are significantly improved. The performance of the proposed method is compared with that of the standard Kalman filter, fractional-order Kalman filter, and unscented Kalman filter. In this work, the root mean square error is used as the performance metric. The proposed method has been tested for different datasets, including traffic videos. Experiments show that RMSE is improved upto $17\%$ by using the proposed modified Kalman filter.

Learning Efficient Data Representations With Orthogonal Sparse Coding

Abstract: We present the learning algorithm orthogonal sparse coding (OSC) to find an orthogonal basis in which a given data set has a maximally sparse representation. OSC is based on stochastic descent by Hebbian-like updates and Gram–Schmidt orthogonalizations, and is motivated by an algorithm that we introduce as the canonical approach (CA). First, we evaluate how well OSC can recover a generating basis from synthetic data. We show that, in contrast to competing methods, OSC can recover the generating basis for quite low and, remarkably, unknown sparsity levels. Moreover, on natural image patches and on images of handwritten digits, OSC learns orthogonal bases that attain significantly sparser representations compared to alternative orthogonal transforms. Furthermore, we demonstrate an application of OSC for image compression by showing that the rate-distortion performance can be improved relative to the JPEG standard. Finally, we demonstrate the state-of-the-art image denoising performance of OSC dictionaries. Our results demonstrate the potential of OSC for feature extraction, data compression, and image denoising, which is due to two important aspects: 1) the learned bases are adapted to the signal class, and 2) the sparse approximation problem can be solved efficiently and exactly.

Simultaneous Temporal Superresolution and Denoising for Cardiac Fluorescence Microscopy

Abstract: Due to low-light emission of fluorescent samples, live fluorescence microscopy imposes a tradeoff between spatio-temporal resolution and signal-to-noise ratio. This can result in images and videos containing motion blur or Poisson-type shot noise, depending on the settings used during acquisition. Here, we propose an algorithm to simultaneously denoise and temporally superresolve movies of repeating microscopic processes that is compatible with any conventional microscopy setup that can achieve imaging at a rate of at least twice that of the fundamental frequency of the process (above 4 frames per second for a 2-Hz process). Our method combines low temporal resolution frames from multiple cycles of a repeating process to reconstruct a denoised, higher temporal resolution image sequence which is the solution to a linear program that maximizes the consistency of the reconstruction with the measurements, under a regularization constraint. This paper describes, in particular, a parallelizable superresolution reconstruction algorithm and demonstrates its application to live cardiac fluorescence microscopy. Using our method, we experimentally show temporal resolution improvement by a factor of 1.6, resulting in a visible reduction of motion blur in both on-sample and off-sample frames.

Combining Inertial Measurements With Blind Image Deblurring Using Distance Transform

Abstract: Camera motion during exposure leads to a blurry image. We propose an image deblurring method that infers the blur kernel by combining the inertial measurement unit (IMU) data that track camera motion with techniques that seek blur cues from the image sensor data. Specifically, we introduce the notion of IMU fidelity cost designed to penalize blur kernels that are unlikely to have yielded the observed IMU measurements. When combined with the image data-based fidelity and regularization terms used by the conventional blind image deblurring techniques, the overall energy function is nonconvex. We solved this nonconvex energy minimization problem by a novel use of distance transform, recovering a blur kernel and sharp image that are consistent with the IMU and image sensor measurements.

Blind X-Ray CT Image Reconstruction From Polychromatic Poisson Measurements

Abstract: We develop a framework for reconstructing images that are sparse in an appropriate transform domain from polychromatic computed tomography (CT) measurements under the blind scenario where the material of the inspected object and incident-energy spectrum are unknown. Assuming that the object that we wish to reconstruct consists of a single material, we obtain a parsimonious measurement-model parameterization by changing the integral variable from photon energy to mass attenuation, which allows us to combine the variations brought by the unknown incident spectrum and mass attenuation into a single unknown mass-attenuation spectrum function; the resulting measurement equation has the Laplace-integral form. The mass-attenuation spectrum is then expanded into basis functions using B-splines of order one. We consider a Poisson noise model and establish conditions for biconvexity of the corresponding log-likelihood (NLL) function with respect to the density-map and mass-attenuation spectrum parameters. We derive a block-coordinate descent algorithm for constrained minimization of a penalized NLL objective function, where penalty terms ensure non-negativity of the mass-attenuation spline coefficients and non-negativity and gradient-map sparsity of the density-map image, imposed using a convex total-variation (TV) norm; the resulting objective function is biconvex. This algorithm alternates between a Nesterovs proximal-gradient (NPG) step and a limited-memory Broyden–Fletcher–Goldfarb–Shanno with box constraints (L-BFGSB) iteration for updating the image and mass-attenuation spectrum parameters, respectively. We prove the Kurdyka–Łojasiewicz property of the objective function, which is important for establishing local convergence of block-coordinate descent schemes in biconvex optimization problems. Our framework applies to other NLLs and signal-sparsity penalties, such as lognormal NLL and ${\ell_1}$ norm of 2-D discrete wavelet transform (DWT) image coefficients. Numerical experiments with simulated and real X-ray CT data demonstrate the performance of the proposed scheme.

An Algorithm Architecture Co-Design for CMOS Compressive High Dynamic Range Imaging

Abstract: Standard image sensors feature dynamic range about 60 to 70 dB while the light flux of natural scenes may be over 120 dB. Most imagers dedicated to address such dynamic ranges, need specific, and large pixels. However, canonical imagers can be used for high dynamic range (HDR) by performing multicapture acquisitions to compensate saturation. This technique is made possible at the expense of the need for large memory requirements and an increase of the overall acquisition time. On the other hand, the implementation of compressive sensing (CS) raises the same issues regarding the modifications of both the pixel and the readout circuitry. Assuming HDR images are sufficiently sparse, CS claims they can be reconstructed from few random linear measurements. A novel CS-based image sensor design is presented in this paper allowing a compressive acquisition without changing the classical pixel design, as well as the overall sensor architecture. In addition to regular CS, HDR CS is enabled thanks to specific time diagrams of the control signals. An alternative nondestructive column-based readout mode constitutes the main change compared to a traditional functioning. The HDR reconstruction, which is also presented in this paper, is based on merging the information of multicapture compressed measurements while taking into account noise sources and nonlinearities introduced by both the proposed acquisition scheme and its practical implementation.

Combining Local Regularity Estimation and Total Variation Optimization for Scale-Free Texture Segmentation

Abstract: Texture segmentation constitutes a standard image processing task, crucial for many applications. The present contribution focuses on the particular subset of scale-free textures and its originality resides in the combination of three key ingredients: First, texture characterization relies on the concept of local regularity; Second, estimation of local regularity is based on new multiscale quantities referred to as wavelet leaders; Third, segmentation from local regularity faces a fundamental bias variance tradeoff. In nature, local regularity estimation shows high variability that impairs the detection of changes, while a posteriori smoothing of regularity estimates precludes from locating correctly changes. Instead, the present contribution proposes several variational problem formulations based on total variation and proximal resolutions that effectively circumvent this tradeoff. Estimation and segmentation performance for the proposed procedures are quantified and compared on synthetic as well as on real-world textures.