Post-processing multiple-frame super-resolution in ultrasound imaging
23 Feb 2012-Proceedings of SPIE (International Society for Optics and Photonics)-Vol. 8320, pp 433-440
TL;DR: It is shown how to overcome the intrinsic limit of super-resolution in this framework by refining the registration part of common multi-frame techniques.
Abstract: High resolution medical ultrasound imaging is an ongoing challenge in many diagnosis applications and can
be achieved by instrumentation. Very few works have investigated ultrasound image resolution enhancement
whereas many works regarded general purpose optical image or video fields. Many algorithms were proposed
within these fields to achieve the "super-resolution" (SR), which consists in merging several low resolution
images to create a higher resolution image. However, the straightforward implementation of such techniques for
ultrasound imaging is unsuccessful, due to the intrinsic nature of ultrasound motions and speckle. We show how
to overcome the intrinsic limit of super-resolution in this framework by refining the registration part of common
multi-frame techniques. Classic super-resolution algorithms were implemented and evaluated using sequences of
ultrasound images. Such methods not only fail to estimate the true elastic motion but also break the speckle
characteristics, resulting in a degradation of the original image. Knowing that a registration error of only 1 pixel
leads to a high-resolution image worse than an interpolation, the registration must be adapted to the framework
of ultrasound imaging. For this purpose, we investigate different motion estimations. The process described
above was evaluated on ultrasound sequences containing up to 15 phantom images with an inclusion scanned
with a 7.5 MHz linear probe. Qualitative improvements were observable as soon as at least 5 low-resolution
images were used. Ultrasound B-mode profiles of radio-frequency lines were studied and the inclusion was more
accurately identified. The Contrast-to-Noise Ratio was increased by approximately 13%.
Citations
More filters
01 Sep 2013
TL;DR: This paper investigates a novel approach based on semi-blind deconvolution formulation and alternating direction method framework in order to perform the ultrasound image restoration task and demonstrates that the technique is more robust to uncertainties in the a priori ultrasonic pulse than classical non-blind decomvolution methods.
Abstract: In the field of ultrasound imaging, resolution enhancement is an up-to-date challenging task. Many device-based approaches have been proposed to overcome the low resolution nature of ultrasound images but very few works deal with post-processing methods. This paper investigates a novel approach based on semi-blind deconvolution formulation and alternating direction method framework in order to perform the ultrasound image restoration task. The algorithm performance is addressed using optical images and synthetic ultrasound data for a various range of criteria. The results demonstrate that our technique is more robust to uncertainties in the a priori ultrasonic pulse than classical non-blind deconvolution methods.
46 citations
Cites methods from "Post-processing multiple-frame supe..."
...…(TV) deconvolution [5] and complex deconvolution framework [6], but more recent works have focused on multiple frame super-resolution (SR) [7, 8], image restoration based on tetrolets shrinkage [9], deconvolution with an inaccurate optical PSF [10], parametric inverse filtering [11, 12], and…...
[...]
TL;DR: Super-resolution forests (SRF), a random forest-based SR method has displayed remarkable high effectiveness and outperformed other methods, should be one potential way to benefit ophthalmologists by obtaining high-resolution FFA images in a clinical setting.
Abstract: Fundus fluorescein angiography (FFA) imaging is a standard diagnostic tool for many retinal diseases such as age-related macular degeneration and diabetic retinopathy High-resolution FFA images facilitate the detection of small lesions such as microaneurysms, and other landmark changes, in the early stages; this can help an ophthalmologist improve a patient’s cure rate However, only low-resolution images are available in most clinical cases Super-resolution (SR), which is a method to improve the resolution of an image, has been successfully employed for natural and remote sensing images To the best of our knowledge, no one has applied SR techniques to FFA imaging so far In this work, we propose a SR method-based pipeline for FFA imaging The aim of this pipeline is to enhance the image quality of FFA by using SR techniques Several SR frameworks including neighborhood embedding, sparsity-based, locally-linear regression and deep learning-based approaches are investigated Based on a clinical FFA dataset collected from Second Affiliated Hospital to Xuzhou Medical University, each SR method is implemented and evaluated for the pipeline to improve the resolution of FFA images As shown in our results, most SR algorithms have a positive impact on the enhancement of FFA images Super-resolution forests (SRF), a random forest-based SR method has displayed remarkable high effectiveness and outperformed other methods Hence, SRF should be one potential way to benefit ophthalmologists by obtaining high-resolution FFA images in a clinical setting
22 citations
Cites methods from "Post-processing multiple-frame supe..."
...In recent years, SR techniques have successfully been extended to medical imaging applications; this provides an important preprocessing step that can improve the image quality of imaging technologies such as ultrasound [26, 27], CT [28], PET [29] and MRI [30–35]....
[...]
TL;DR: This work investigates motion estimation methods adapted to US imaging and shows how to overcome the limit of SR in this framework by refining the registration part of common multiframe techniques.
18 citations
Cites background or methods from "Post-processing multiple-frame supe..."
...The straightforward implementation of such algorithms for US imaging was unsuccessful, due to the intrinsic nature of tissue elastic motions, speckle and the point spread function (PSF) [11]....
[...]
...A third deblurring step (not addressed in this paper, see [18,19]) can be added in order to enhance the output image Jðx; yÞ, depending on the observation model and the available a priori information about the PSF. Note that in the proposed framework, the motion estimation stage is the key element of the SR image reconstruction and will hence almost completely determine the quality of the resulting image....
[...]
...The CNR was successively computed using one LR input image and the three HR images based on bicubic interpolation, classic SR as reported in [11] and our method (see the corresponding images in Fig....
[...]
...We have recently shown in [11] that classical motion estimation, usually involved in multiframe SR frameworks (see [10] and references therein), provide poor results when performed...
[...]
...Indeed, these methods fail to estimate the true elastic motion and therefore break the speckle characteristics, resulting in an image degradation [11]....
[...]
TL;DR: Qualitative analysis of the reconstructed images confirms that the proposed super-resolution technique achieves much better quality HR images than other methods in terms of the human visual system and there is a statistically significant difference between it and others.
Abstract: Ultrasound offers a safe, non-invasive, and inexpensive way of imaging. However, due to its natural intrinsic imaging characteristics, it produces poor quality images with low resolution (LR) compared to other medical imaging modalities. Various image enhancement techniques have been extensively studied to overcome these shortcomings. Super-resolution (SR) is one of these methods, which endeavor to obtain high resolution (HR) images from LR images while enlarging them. Numerous studies have already utilized different SR techniques in various stages of ultrasound imaging (USI). Unlike other studies, which aimed at obtaining SR in the pre-processing phase or early stages of the post-processing phase of USI, we achieved SR on B-mode ultrasound images, which is the last stage of USI. We constructed a deep convolutional neural network (CNN) and trained it with a very large dataset of B-mode ultrasound images for the scale factors 2, 3, 4, and 8. We evaluated the performance of our proposed model quantitatively with eight image quality measures. The quantitative results revealed that our algorithm is much more successful than other methods at each magnification factor. Furthermore, we also verified that there is a statistically significant difference between our approach and others. Besides, qualitative analysis of the reconstructed images also confirms that it produces much better quality HR images than other methods in terms of the human visual system.
13 citations
Cites methods from "Post-processing multiple-frame supe..."
...A multi-frame SR task was experimented in [39]....
[...]
Dissertation•
01 Jan 2013
TL;DR: In this paper, deux approches prennent en compte, suivant leur disponibilite, certaines informations a priori sur les conditions d'acquisition comme la reponse impulsionnelle spatiale du systeme.
Abstract: L'imagerie ultrasonore est une modalite d'imagerie medicale couramment utilisee pour l'etablissement de diagnostics de therapie ou de suivi divers (croissance du fœtus, detection de certains cancers, assistance a la realisation d'actes therapeutiques). Si cette modalite dispose de nombreux avantages comme son innocuite, sa facilite d'utilisation et son faible cout, elle souffre cependant d'une resolution spatiale limitee quand on la compare a d'autres modalites comme l'imagerie par resonance magnetique. L'amelioration de la resolution des images ultrasonores est un defi de taille et de tres nombreux travaux ont depuis longtemps explore des approches instrumentales portant sur l'optimisation du dispositif d'acquisition. L'imagerie echographique haute resolution permet ainsi d'atteindre cet objectif a l'aide de sondes particulieres mais se trouve aujourd'hui confrontee a des limitations d'ordre physique et technologique. L'objet de cette these est d'adopter une strategie de post-traitement afin de contourner ces contraintes inherentes aux approches instrumentales. Dans ce contexte, nous presentons deux approches pour l'amelioration de la resolution des images ultrasonores, selon que les donnees disponibles prennent la forme d'une sequence d'images ou d'une image unique. Dans le premier cas, l'adaptation d'une technique d'estimation du mouvement originellement proposee pour l'elastographie nous permet d'etablir un cadre de reconstruction haute resolution efficace dedie a la modalite qui nous interesse. Cette approche est evaluee a l'aide d'une simulation realiste d'images ultrasonores avant d'etre appliquee a des donnees in vivo. Nous proposons ensuite, dans le cadre du traitement d'une seule image, deux methodes de deconvolution rapide pour l'amelioration de la resolution. Ces approches prennent en compte, suivant leur disponibilite, certaines informations a priori sur les conditions d'acquisition comme la reponse impulsionnelle spatiale du systeme. Les resultats sont caracterises dans un premier temps a l'aide de donnees synthetiques et sont ensuite valides sur des images in vivo
8 citations
Cites methods from "Post-processing multiple-frame supe..."
...L’adaptation d’un algorithme d’estimation du mouvement originellement développé pour l’élastographie nous permet d’établir un cadre de reconstruction haute résolution efficace dédié à l’imagerie ultrasonore [Morin et al., 2012b]....
[...]
...Un cadre algorithmique robuste et rapide issu des développements récents en optimisation efficace est mis en place afin d’estimer l’image ultrasonore haute résolution à partir d’une image observée à une résolution inférieure [Morin et al., 2012a]....
[...]
References
More filters
TL;DR: A review of recent as well as classic image registration methods to provide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas.
6,842 citations
TL;DR: The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts to present the technical review of various existing SR methodologies which are often employed.
Abstract: A new approach toward increasing spatial resolution is required to overcome the limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an high-resolution (HR) image (or sequence) from observed multiple low-resolution (LR) images. Such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement. In this article, we use the term "SR image reconstruction" to refer to a signal processing approach toward resolution enhancement because the term "super" in "super resolution" represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts. To this purpose, we present the technical review of various existing SR methodologies which are often employed. Before presenting the review of existing SR algorithms, we first model the LR image acquisition process.
3,491 citations
"Post-processing multiple-frame supe..." refers methods in this paper
...works regarded this resolution enhancement aspect within general purpose optical image or video fields.(5) Various algorithms were designed to perform this resolution enhancement task, such as iterative back-projection,(6) minimization of a regularized cost functional through standard(7) or improved(8) methods, projection onto convex sets (POCS)....
[...]
TL;DR: In this paper, the relative displacements in image sequences are known accurately, and some knowledge of the imaging process is available, and the proposed approach is similar to back-projection used in tomography.
2,081 citations
19 May 1992
TL;DR: In this paper, a hierarchical estimation framework for the computation of diverse representations of motion information is described, which includes a global model that constrains the overall structure of the motion estimated, a local model that is used in the estimation process, and a coarse-fine refinement strategy.
Abstract: This paper describes a hierarchical estimation framework for the computation of diverse representations of motion information. The key features of the resulting framework (or family of algorithms) are a global model that constrains the overall structure of the motion estimated, a local model that is used in the estimation process, and a coarse-fine refinement strategy. Four specific motion models: affine flow, planar surface flow, rigid body motion, and general optical flow, are described along with their application to specific examples.
1,501 citations
Book•
05 Dec 2013
TL;DR: Diagnostic Ultrasound Imaging provides a unified description of the physical principles of ultrasound imaging, signal processing, systems and measurements that enable practicing engineers, students and clinical professionals to understand the essential physics and signal processing techniques behind modern imaging systems.
Abstract: Diagnostic Ultrasound Imaging provides a unified description of the physical principles of ultrasound imaging, signal processing, systems and measurements. This comprehensive reference is a core resource for both graduate students and engineers in medical ultrasound research and design. With continuing rapid technological development of ultrasound in medical diagnosis, it is a critical subject for biomedical engineers, clinical and healthcare engineers and practitioners, medical physicists, and related professionals in the fields of signal and image processing. The book contains 17 new and updated chapters covering the fundamentals and latest advances in the area, and includes four appendices, 450 figures (60 available in color on the companion website), and almost 1,500 references. In addition to the continual influx of readers entering the field of ultrasound worldwide who need the broad grounding in the core technologies of ultrasound, this book provides those already working in these areas with clear and comprehensive expositions of these key new topics as well as introductions to state-of-the-art innovations in this field. * Enables practicing engineers, students and clinical professionals to understand the essential physics and signal processing techniques behind modern imaging systems as well as introducing the latest developments that will shape medical ultrasound in the future* Suitable for both newcomers and experienced readers, the practical, progressively organized applied approach is supported by hands-on MATLAB code and worked examples that enable readers to understand the principles underlying diagnostic and therapeutic ultrasound* Covers the new important developments in the use of medical ultrasound: elastography and high-intensity therapeutic ultrasound. Many new developments are comprehensively reviewed and explained, including aberration correction, acoustic measurements, acoustic radiation force imaging, alternate imaging architectures, bioeffects: diagnostic to therapeutic, Fourier transform imaging, multimode imaging, plane wave compounding, research platforms, synthetic aperture, vector Doppler, transient shear wave elastography, ultrafast imaging and Doppler, functional ultrasound and viscoelastic models
1,170 citations