We give a methodology-oriented perspective on directional image analysis and rotation-invariant processing. We review the state of the art in the field and make connections with recent mathematical developments in functional analysis and wavelet theory. We unify our perspective within a common framework using operators. The intent is to provide image-processing methods that can be deployed in algorithms that analyze biomedical images with improved rotation invariance and high directional sensitivity. We start our survey with classical methods such as directional-gradient and the structure tensor. Then, we discuss how these methods can be improved with respect to robustness, invariance to geometric transformations (with a particular interest in scaling), and computation cost. To address robustness against noise, we move forward to higher degrees of directional selectivity and discuss Hessian-based detection schemes. To present multiscale approaches, we explain the differences between Fourier filters, directional wavelets, curvelets, and shearlets. To reduce the computational cost, we address the problem of matching directional patterns by proposing steerable filters, where one might perform arbitrary rotations and optimizations without discretizing the orientation. We define the property of steerability and give an introduction to the design of steerable filters. We cover the spectrum from simple steerable filters through pyramid schemes up to steerable wavelets. We also present illustrations on the design of steerable wavelets and their application to pattern recognition.

/pdf/transforms-and-operators-for-directional-bioimage-analysis-a-cz4ketezr8.pdf

Transforms and Operators for Directional Bioimage Analysis: A Survey

This paper presents a robust and fully automatic filter-based approach for retinal vessel segmentation. We propose new filters based on 3D rotating frames in so-called orientation scores, which are functions on the Lie-group domain of positions and orientations $\mathbb {R}^{2} \rtimes S^{1}$ . By means of a wavelet-type transform, a 2D image is lifted to a 3D orientation score, where elongated structures are disentangled into their corresponding orientation planes. In the lifted domain $\mathbb {R}^{2} \rtimes S^{1}$ , vessels are enhanced by means of multi-scale second-order Gaussian derivatives perpendicular to the line structures. More precisely, we use a left-invariant rotating derivative (LID) frame, and a locally adaptive derivative (LAD) frame. The LAD is adaptive to the local line structures and is found by eigensystem analysis of the left-invariant Hessian matrix (computed with the LID). After multi-scale filtering via the LID or LAD in the orientation score domain, the results are projected back to the 2D image plane giving us the enhanced vessels. Then a binary segmentation is obtained through thresholding. The proposed methods are validated on six retinal image datasets with different image types, on which competitive segmentation performances are achieved. In particular, the proposed algorithm of applying the LAD filter on orientation scores (LAD-OS) outperforms most of the state-of-the-art methods. The LAD-OS is capable of dealing with typically difficult cases like crossings, central arterial reflex, closely parallel and tiny vessels. The high computational speed of the proposed methods allows processing of large datasets in a screening setting.

Robust Retinal Vessel Segmentation via Locally Adaptive Derivative Frames in Orientation Scores

We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equivariant convolutions utilize kernels which are parameterized as a linear combination of a complete steerable kernel basis, which is derived analytically in this paper. We prove that equivariant convolutions are the most general equivariant linear maps between fields over ℝ3. Our experimental results confirm the effectiveness of 3D Steerable CNNs for the problem of amino acid propensity prediction and protein structure classification, both of which have inherent SE(3) symmetry.

/pdf/3d-steerable-cnns-learning-rotationally-equivariant-features-fav2ipjbnj.pdf

3D steerable CNNs: learning rotationally equivariant features in volumetric data

A rigid-motion scattering computes adaptive invariants along translations and rotations, with a deep convolutional network. Convolutions are calculated on the rigid-motion group, with wavelets defined on the translation and rotation variables. It preserves joint rotation and translation information, while providing global invariants at any desired scale. Texture classification is studied, through the characterization of stationary processes from a single realization. State-of-the-art results are obtained on multiple texture data bases, with important rotation and scaling variabilities.

Rigid-Motion Scattering for Texture Classification.

Endovascular techniques have been embraced as a minimally-invasive treatment approach within different disciplines of interventional radiology and cardiology. The current practice of endovascular procedures, however, is limited by a number of factors including exposure to high doses of X-ray radiation, limited 3D imaging, and lack of contact force sensing from the endovascular tools and the vascular anatomy. More recently, advances in steerable catheters and development of master/slave robots have aimed to improve these practices by removing the operator from the radiation source and increasing the precision and stability of catheter motion with added degrees-of-freedom. Despite their increased application and a growing research interest in this area, many such systems have been designed without considering the natural manipulation skills and ergonomic preferences of the operators. Existing studies on tool interactions and natural manipulation skills of the operators are limited. In this manuscript, new technical developments in different aspects of robotic endovascular intervention including catheter instrumentation, intra-operative imaging and navigation techniques, as well as master/slave based robotic catheterization platforms are reviewed. We further address emerging trends and new research opportunities towards more widespread clinical acceptance of robotically assisted endovascular technologies.

Current and emerging robot-assisted endovascular catheterization technologies: a review.

We provide the explicit solutions of linear, left-invariant, diffusion equations and the corresponding resolvent equations on the 2D-Euclidean motion group SE(2) = R^2 x T. These parabolic equations are forward Kolmogorov equations for well-known stochastic processes for contour enhancement and contour completion. The solutions are given by group convolution with the corresponding Green's functions. In earlier work we have solved the forward Kolmogorov equations (or Fokker-Planck equations) for stochastic processes on contour completion. Here we mainly focus on the forward Kolmogorov equations for contour enhancement processes which do not include convection. We derive explicit formulas for the Green's functions (i.e., the heat kernels on SE(2)) of the left-invariant partial differential equations related to the contour enhancement process. By applying a contraction we approximate the left-invariant vector fields on SE(2) by left-invariant generators of a Heisenberg group, and we derive suitable approximations of the Green's functions. The exact Green's functions are used in so-called collision distributions on SE(2), which are the product of two left-invariant resolvent diffusions given an initial distribution on SE(2). We use the left-invariant evolution processes for automated contour enhancement in noisy medical image data using a so-called orientation score, which is obtained from a grey-value image by means of a special type of unitary wavelet transformation. Here the real part of the (invertible) orientation score serves as an initial condition in the collision distribution.

/pdf/left-invariant-parabolic-evolutions-on-se-2-and-contour-4chwa7tb7j.pdf

Left-invariant parabolic evolutions on SE(2) and contour enhancement via invertible orientation scores. Part I: Linear left-invariant diffusion equations on SE(2)

HARDI (High Angular Resolution Diffusion Imaging) is a recent magnetic resonance imaging (MRI) technique for imaging water diffusion processes in fibrous tissues such as brain white matter and muscles. In this article we study left-invariant diffusion on the group of 3D rigid body movements (i.e. 3D Euclidean motion group) SE(3) and its application to crossing-preserving smoothing of HARDI images. The linear left-invariant (convection-)diffusions are forward Kolmogorov equations of Brownian motions on the space of positions and orientations in 3D embedded in SE(3) and can be solved by ?3 ? S 2-convolution with the corresponding Green's functions. We provide analytic approximation formulas and explicit sharp Gaussian estimates for these Green's functions. In our design and analysis for appropriate (nonlinear) convection-diffusions on HARDI data we explain the underlying differential geometry on SE(3). We write our left-invariant diffusions in covariant derivatives on SE(3) using the Cartan connection. This Cartan connection has constant curvature and constant torsion, and so have the exponential curves which are the auto-parallels along which our left-invariant diffusion takes place. We provide experiments of our crossing-preserving Euclidean-invariant diffusions on artificial HARDI data containing crossing-fibers.

/pdf/left-invariant-diffusions-on-the-space-of-positions-and-38dfj0k2um.pdf

Left-Invariant Diffusions on the Space of Positions and Orientations and their Application to Crossing-Preserving Smoothing of HARDI images

Many image processing problems require the enhancement of crossing elongated structures. These problems cannot easily be solved by commonly used coherence-enhancing diffusion methods. Therefore, we propose a method for coherence-enhancing diffusion on the invertible orientation score of a 2D image. In an orientation score, the local orientation is represented by an additional third dimension, ensuring that crossing elongated structures are separated from each other. We consider orientation scores as functions on the Euclidean motion group, and use the group structure to apply left-invariant diffusion equations on orientation scores. We describe how we can calculate regularized left-invariant derivatives, and use the Hessian to estimate three descriptive local features: curvature, deviation from horizontality, and orientation confidence. These local features are used to adapt a nonlinear coherence-enhancing, crossing-preserving, diffusion equation on the orientation score. We propose two explicit finite-difference schemes to apply the nonlinear diffusion in the orientation score and provide a stability analysis. Experiments on both artificial and medical images show that preservation of crossings is the main advantage compared to standard coherence-enhancing diffusion. The use of curvature leads to improved enhancement of curves with high curvature. Furthermore, the use of deviation from horizontality makes it feasible to reduce the number of sampled orientations while still preserving crossings.

/pdf/crossing-preserving-coherence-enhancing-diffusion-on-36kf05nw4q.pdf

Crossing-Preserving Coherence-Enhancing Diffusion on Invertible Orientation Scores

By means of a special type of wavelet unitary transform we construct an orientation score from a grey-value image. This orientation score is a complex-valued function on the 2D Euclidean motion group SE(2) and gives us explicit information on the presence of local orientations in an image. As the transform between image and orientation score is unitary we can relate operators on images to operators on orientation scores in a robust manner. Here we consider nonlinear adaptive diffusion equations on these invertible orientation scores. These nonlinear diffusion equations lead to clear improvements of the celebrated standard "coherence enhancing diffusion" equations on images as they can enhance images with crossing contours. Here we employ differential geometry on SE(2) to align the diffusion with optimized local coordinate systems attached to an orientation score, allowing us to include local features such as adaptive curvature in our diffusions.

/pdf/left-invariant-parabolic-evolutions-on-se-2-and-contour-5bmvnvlxic.pdf

Left-invariant parabolic evolutions on SE(2) and contour enhancement via invertible orientation scores. Part II: Non-linear left-invariant diffusions on invertible orientation scores

Erik Franken

Papers

Left-invariant parabolic evolutions on SE(2) and contour enhancement via invertible orientation scores. Part I: Linear left-invariant diffusion equations on SE(2)

Left-Invariant Diffusions on the Space of Positions and Orientations and their Application to Crossing-Preserving Smoothing of HARDI images

Crossing-Preserving Coherence-Enhancing Diffusion on Invertible Orientation Scores

Left-invariant parabolic evolutions on SE(2) and contour enhancement via invertible orientation scores. Part II: Non-linear left-invariant diffusions on invertible orientation scores

Crossing-preserving coherence-enhancing diffusion on invertible orientation scores