It was recently demonstrated that one can perform fast nonlocal means (NLM) denoising of one-dimensional (1-D) signals using a method called lifting. The cost of lifting is independent of the patch length, which dramatically reduces the run-time for large patches. Unfortunately, it is difficult to directly extend lifting for NLM denoising of images. To bypass this, the authors proposed a separable approximation in which the image rows and columns are filtered using lifting. The overall algorithm is significantly faster than NLM, and the results are comparable in terms of PSNR. However, the separable processing often produces vertical and horizontal stripes in the image. This problem was previously addressed using a bilateral filter-based postsmoothing, which was effective in removing some of the stripes. We demonstrate that stripes can be mitigated in the first place simply by involving the neighboring rows (or columns) in the filtering. In other words, we use a two-dimensional (2-D) search (similar to NLM), while still using 1-D patches (as in the previous proposal). The innovation is in the observation that one can use lifting for performing 2-D searches. The proposed approach produces artifact-free images, whose quality and PSNR are comparable to NLM, while being significantly faster. (C) 2017 SPIE and IS&T

Artifact reduction for separable nonlocal means

Noise removal is a fundamental requirement for many image processing applications. Its goal is filtering noise from images while preserving integrity of fine details and edges. In order to assess the quality of image denoising algorithms, it is of paramount importance to use suitable metrics. In other words, the use of an appropriate Image Quality Assessment (IQA) procedure is a key factor for selecting the most suitable image denoising filter and, hence, it has a major impact on the quality of the final image. This paper demonstrates that currently used IQA metrics are not suitable for selecting optimal image denoising filters when edge preservation must be evaluated, because they provide quality figures for the entire image, without specifically addressing edges. From this fact, a method for selecting the optimal filter in this context is proposed, based on minimizing a novel metric that accounts for filter response in edges. Smoothing and sharpening algorithms have been applied to different noisy images and then evaluated using popular metrics and the proposed one. Results show that the new metric is more suitable to assess final image quality when edge deterioration effects have to be accounted for, while offering a simple and understandable approach.

A quantitative method for selecting denoising filters, based on a new edge-sensitive metric

It was recently demonstrated [J. Electron. Imaging, 25(2), 2016] that one can perform fast non-local means (NLM) denoising of one-dimensional signals using a method called lifting. The cost of lifting is independent of the patch length, which dramatically reduces the run-time for large patches. Unfortunately, it is difficult to directly extend lifting for non-local means denoising of images. To bypass this, the authors proposed a separable approximation in which the image rows and columns are filtered using lifting. The overall algorithm is significantly faster than NLM, and the results are comparable in terms of PSNR. However, the separable processing often produces vertical and horizontal stripes in the image. This problem was previously addressed by using a bilateral filter-based post-smoothing, which was effective in removing some of the stripes. In this letter, we demonstrate that stripes can be mitigated in the first place simply by involving the neighboring rows (or columns) in the filtering. In other words, we use a two-dimensional search (similar to NLM), while still using one-dimensional patches (as in the previous proposal). The novelty is in the observation that one can use lifting for performing two-dimensional searches. The proposed approach produces artifact-free images, whose quality and PSNR are comparable to NLM, while being significantly faster.

Artifact reduction for separable non-local means

Author(s): Chacko, Nikhil | Advisor(s): Liebling, Michael | Abstract: Optical microscopy offers the unique possibility to study living samples under conditions akin to their native state. However, the technique is not void of inherent problems such as optical blur due to light diffraction, contamination with out-of-focus light from adjacent focal planes, and spherical aberrations. Furthermore, with a dearth of techniques that are capable of imaging multiple focal sections in quick succession, the multi-dimensional capture of dynamically changing samples remains a challenge of its own. Computational techniques that use auxiliary knowledge about the imaging system and the sample to mitigate these problems are hence of great interest in optical microscopy.The first part of this thesis deals with the design of a discrete model to characterize light propagation. Following the scalar diffraction theory in optics, we propose a discrete algorithm, based on generalized sampling theory, to reverse the coherent diffraction process via back propagation. The algorithm consists of a wavelet-based model for the spherical waves emanating from the object of interest and an optimized multi-rate filtering protocol for reconstruction from the diffraction data recorded by non-ideal detectors. The second part of this thesis describes a spatial registration tool designed for multi-view microscopy. Here, the imaged sample is rotated about a lateral axis for the acquisition of multiple 3D datasets from different views in order to subsequently alleviate the severe axial blur found in each such dataset. Automatic algorithms that only rely on maximizing pixel-based similarity provide poor results in such applications owing to the anisotropic point-spread-function (PSF) of optical microscopes. We propose a pyramid-based spatial registration algorithm that re-blurs the multi-view datasets with transformed forms of the PSF in order to make them comparable, before maximizing their pixel-based similarity for registration.The third part of this thesis describes a fast converging iterative multi-view deconvolution technique that can be applied to the spatially registered forms of the 3D datasets acquired using multi-view microscopy. Our sparsity based algorithm solves a non-linear objective function to jointly deconvolve and fuse the multi-view datasets to finally produce a single deblurred 3D result that has nearly isotropic spatial resolution.The fourth part of this thesis addresses problems due to spherical aberrations encountered during the imaging of thick samples in optical microscopy. The depth-varying nature of the optical blur found in such cases renders fast and efficient shift-invariant deconvolution techniques to be inapplicable. Here, we propose a fast iterative-shrinkage-thresholding shift-variant 3D deconvolution method that uses depth-dependent PSFs to reconstruct a 3D deblurred form of the imaged thick specimen. The final part of this thesis describes a non-rigid temporal registration tool that aids in the multi-dimensional imaging of quasi-periodic processes such as cardiac cycles. We propose a variant of dynamic time warping that is capable of both temporally warping and wrapping an input sequence by allowing for jump discontinuities in the non-linear temporal alignment function akin to those found in wrapped phase functions. This work provides a new set of tools for spatio-temporal reconstruction in optical microscopy and we anticipate them to be useful for a wide range of problems in practice.

Spatio-Temporal Reconstruction Techniques for Optical Microscopy

Non-local means (NLM) noise reduction is effective yet computationally intensive, since each output pixel is a weighted average of all input pixels. Most implementations therefore restrict the scope of the average to a smaller neighborhood around each pixel, which limits the method's full potential. Here we propose to apply NLM to reduce the noise in fluorescence microscopy image sequences of the beating heart, whose quasi-repeatable pattern produces multiple realizations of similar image patches in different heart beats. We propose to restrict the averaging from the entire sequence to a subset of globally similar frames across multiple heart beats. Using a high-SNR brightfield microscopy image sequence of a beating embryonic zebrafish heart that we artificially corrupt with noise, we illustrate the benefits of selecting non-adjacent frames rather than the immediate temporal neighborhood in the vicinity of the pixel being denoised. The image quality of our NLM approach is also better than that obtained by directly computing the sample median of matching frames over multiple heartbeats, a commonly used method. Finally, we demonstrate the applicability of our proposed scheme to low-intensity fluorescence images of the embryonic zebrafish heart.

John M. Batikian

Papers

Multicycle non-local means denoising of cardiac image sequences