Lisa Gottesfeld Brown

Bio: Lisa Gottesfeld Brown is an academic researcher from Columbia University. The author has contributed to research in topics: Image registration & Singular value. The author has an hindex of 3, co-authored 4 publications receiving 4632 citations.

A survey of image registration techniques

Abstract: Registration is a fundamental task in image processing used to match two or more pictures taken, for example, at different times, from different sensors, or from different viewpoints. Virtually all large systems which evaluate images require the registration of images, or a closely related operation, as an intermediate step. Specific examples of systems where image registration is a significant component include matching a target with a real-time image of a scene for target recognition, monitoring global land usage using satellite images, matching stereo images to recover shape for autonomous navigation, and aligning images from different medical modalities for diagnosis.Over the years, a broad range of techniques has been developed for various types of data and problems. These techniques have been independently studied for several different applications, resulting in a large body of research. This paper organizes this material by establishing the relationship between the variations in the images and the type of registration techniques which can most appropriately be applied. Three major types of variations are distinguished. The first type are the variations due to the differences in acquisition which cause the images to be misaligned. To register images, a spatial transformation is found which will remove these variations. The class of transformations which must be searched to find the optimal transformation is determined by knowledge about the variations of this type. The transformation class in turn influences the general technique that should be taken. The second type of variations are those which are also due to differences in acquisition, but cannot be modeled easily such as lighting and atmospheric conditions. This type usually effects intensity values, but they may also be spatial, such as perspective distortions. The third type of variations are differences in the images that are of interest such as object movements, growths, or other scene changes. Variations of the second and third type are not directly removed by registration, but they make registration more difficult since an exact match is no longer possible. In particular, it is critical that variations of the third type are not removed. Knowledge about the characteristics of each type of variation effect the choice of feature space, similarity measure, search space, and search strategy which will make up the final technique. All registration techniques can be viewed as different combinations of these choices. This framework is useful for understanding the merits and relationships between the wide variety of existing techniques and for assisting in the selection of the most suitable technique for a specific problem.

Landmark-based 3D fusion of SPECT and CT images

Abstract: In this paper we present interactive visualization procedures for registration of SPECT and CT images based on landmarks. Because of the poor anatomic detail available in many SPECT images, registration of SPECT images with other modalities often requires the use of external markers. These markers may correspond to anatomic structures identifiable in the other modality image. In this work, we present a method to nonrigidly register SPECT and CT images based on automatic marker localization and interactive anatomic localization using 3D surface renderings of skin. The images are registered in 3D by fitting low order polynomials which are constrained to be near rigid. The method developed here exploits 3D information to attain greater accuracy and reduces the amount of time needed for expert interaction.

Registration of multi-modal medical images: exploiting sensor relationships

Abstract: This thesis addresses the problem of registering images from different medical sensors Although significant progress has been made for general-purpose registration methods, this thesis shows that to achieve the accuracy necessary for clinical purposes, multi-modal medical applications must incorporate specific information regarding the relationship between sensing devices To study the ability to exploit sensor models for multi-modal medical registration, the primary application considered is the registration of planar film radiographs with X-ray Computed Tomography (CT) This includes a study of the accuracy of localization from multiple radiographs, the calibration of radiograph and CT data, the development of a sensor relationship model and a corresponding methodology for the registration of planar film radiographs with CT measurements In addition, a similar strategy for the registration of B-scan ultrasound and CT data is discussed Previous methods applied to the problem of CT-radiograph registration rely on determining the correspondence between occluding contours of the 3D surface in the CT data with 2D contours in the projection image These methods implicitly assume that the correspondence is accurate, ignoring fundamental nonlinear differences in the underlying measurements In contrast, our emphasis is to directly exploit the relationship between imaging devices This is performed by registering radiograph data with intensity-corrected simulated radiograph data derived from CT measurements We show that by exploiting the physical relationship between CT and radiograph measurements we can significantly improve registration accuracy Concomitantly, we detail the relationship between CT and radiograph measurements and the primary factors influencing discrepancies between simulated and real radiograph data

Motion segmentation using singular value decomposition

Abstract: This paper presents a method for the segmentation of multiple motions in a scene using the singular value decomposition of a feature track matrix. It is shown that motions can be separated using the right singular vectors associated with the nonzero singular values. This is based on the relationship between the right singular vectors and the principal components of the covariance matrix of the tracks. Furthermore, under general assumptions, the number of numerically nonzero singular values can be used to determine the number of motions. This can be used to derive a relationship between a good segmentation, the number of nonzero singular values in the input and the sum of the number of nonzero singular values in the segments. The approach is demonstrated on real and synthetic examples and a study of the robustness of the method is given. The paper ends with a critical analysis of the approach.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A taxonomy and evaluation of dense two-frame stereo correspondence algorithms

Abstract: Stereo matching is one of the most active research areas in computer vision. While a large number of algorithms for stereo correspondence have been developed, relatively little work has been done on characterizing their performance. In this paper, we present a taxonomy of dense, two-frame stereo methods designed to assess the different components and design decisions made in individual stereo algorithms. Using this taxonomy, we compare existing stereo methods and present experiments evaluating the performance of many different variants. In order to establish a common software platform and a collection of data sets for easy evaluation, we have designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can be easily extended to include new algorithms. We have also produced several new multiframe stereo data sets with ground truth, and are making both the code and data sets available on the Web.

Multimodality image registration by maximization of mutual information

Abstract: A new approach to the problem of multimodality medical image registration is proposed, using a basic concept from information theory, mutual information (MI), or relative entropy, as a new matching criterion. The method presented in this paper applies MI to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned. Maximization of MI is a very general and powerful criterion, because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved. The accuracy of the MI criterion is validated for rigid body registration of computed tomography (CT), magnetic resonance (MR), and photon emission tomography (PET) images by comparison with the stereotactic registration solution, while robustness is evaluated with respect to implementation issues, such as interpolation and optimization, and image content, including partial overlap and image degradation. Our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications.

Computer Vision: Algorithms and Applications

Abstract: Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

Super-resolution image reconstruction: a technical overview

Abstract: A new approach toward increasing spatial resolution is required to overcome the limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an high-resolution (HR) image (or sequence) from observed multiple low-resolution (LR) images. Such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement. In this article, we use the term "SR image reconstruction" to refer to a signal processing approach toward resolution enhancement because the term "super" in "super resolution" represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts. To this purpose, we present the technical review of various existing SR methodologies which are often employed. Before presenting the review of existing SR algorithms, we first model the LR image acquisition process.