Three-dimensional surface reconstruction using optical flow for medical imaging
TL;DR: The experimental results suggest that the proposed surface rendering method is suitable for the reconstruction of 3-D models from ultrasound medical images as well as other computed tomograms.
Abstract: The recovery of a three-dimensional (3-D) model from a sequence of two-dimensional (2-D) images is very useful in medical image analysis. Image sequences obtained from the relative motion between the object and the camera or the scanner contain more 3-D information than a single image. Methods to visualize the computed tomograms can be divided into two approaches: the surface rendering approach and the volume rendering approach. In this paper, a new surface rendering method using optical flow is proposed. Optical flow is the apparent motion in the image plane produced by the projection of real 3-D motion onto the 2-D image. The 3-D motion of an object can be recovered from the optical-flow field using additional constraints. By extracting the surface information from 3-D motion, it is possible to obtain an accurate 3-D model of the object. Both synthetic and real image sequences have been used to illustrate the feasibility of the proposed method. The experimental results suggest that the proposed method is suitable for the reconstruction of 3-D models from ultrasound medical images as well as other computed tomograms.
Citations
More filters
Proceedings Article•
[...]
TL;DR: A scheme is developed for classifying the types of motion perceived by a humanlike robot and equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented.
Abstract: A scheme is developed for classifying the types of motion perceived by a humanlike robot. It is assumed that the robot receives visual images of the scene using a perspective system model. Equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented. >
2,000Â citations
Patent•
11 Jan 2000
TL;DR: In this article, a method for determining a characteristic of a sample of a material by the interaction of electromagnetic radiation with the sample is described. Butts et al. use an optical assembly and a protective barrier to prevent contamination of the optical assembly by body tissue.
Abstract: Methods and apparatus are provided for determining a characteristic of a sample of a material by the interaction of electromagnetic radiation with the sample. The apparatus includes an optical assembly and a protective barrier. The optical assembly sequentially illuminates a plurality of volume elements in the sample with an intensity distribution in the sample that drops off substantially monotonically from a first region in a first optical path and collects electromagnetic radiation emanating from each of the volume elements. The optical assembly collects the electromagnetic radiation emanating from each of the volume elements with a collected distribution that drops off substantially monotonically from a second region in a second optical path. The first and second regions at least partially overlap in each of the volume elements. The optical assembly can be configured as a probe, to be directed to the evaluation of a sample of a biological material. A protective barrier can be disposed between the optical assembly and a body tissue, to prevent contamination of the optical assembly by said body tissue.
297Â citations
Posted Content•
TL;DR: The promising recognition result suggests that the proposed method can optimally describe the significant micro-expression details, and is the first work that performs cross-dataset validation on three databases in this domain.
Abstract: When a person attempts to conceal an emotion, the genuine emotion is manifest as a micro-expression. Exploration of automatic facial micro-expression recognition systems is relatively new in the computer vision domain. This is due to the difficulty in implementing optimal feature extraction methods to cope with the subtlety and brief motion characteristics of the expression. Most of the existing approaches extract the subtle facial movements based on hand-crafted features. In this paper, we address the micro-expression recognition task with a convolutional neural network (CNN) architecture, which well integrates the features extracted from each video. A new feature descriptor, Optical Flow Features from Apex frame Network (OFF-ApexNet) is introduced. This feature descriptor combines the optical ow guided context with the CNN. Firstly, we obtain the location of the apex frame from each video sequence as it portrays the highest intensity of facial motion among all frames. Then, the optical ow information are attained from the apex frame and a reference frame (i.e., onset frame). Finally, the optical flow features are fed into a pre-designed CNN model for further feature enhancement as well as to carry out the expression classification. To evaluate the effectiveness of OFF-ApexNet, comprehensive evaluations are conducted on three public spontaneous micro-expression datasets (i.e., SMIC, CASME II and SAMM). The promising recognition result suggests that the proposed method can optimally describe the significant micro-expression details. In particular, we report that, in a multi-database with leave-one-subject-out cross-validation experimental protocol, the recognition performance reaches 74.60% of recognition accuracy and F-measure of 71.04%. We also note that this is the first work that performs cross-dataset validation on three databases in this domain.
104Â citations
Cites background from "Three-dimensional surface reconstru..."
...Apart from LBPTOP, optical flow [35] is one of the most popular feature extractors, as it has been very successful in a variety of computer vision tasks, such as action recognition [36], face tracking [37], medical image reconstruction [38]....
[...]
TL;DR: In this paper, the optical flow features from Apex frame network (OFF-ApexNet) is proposed for facial micro-expression recognition with a CNN architecture, which well integrates the features extracted from each video.
Abstract: When a person attempts to conceal an emotion, the genuine emotion is manifest as a micro-expression. Exploration of automatic facial micro-expression recognition systems is relatively new in the computer vision domain. This is due to the difficulty in implementing optimal feature extraction methods to cope with the subtlety and brief motion characteristics of the expression. Most of the existing approaches extract the subtle facial movements based on hand-crafted features. In this paper, we address the micro-expression recognition task with a convolutional neural network (CNN) architecture, which well integrates the features extracted from each video. We introduce the Optical Flow Features from Apex frame Network (OFF-ApexNet). This is a new feature descriptor that combines the optical flow guided context with the CNN. Firstly, we obtain the location of the apex frame from each video sequence as it portrays the highest intensity of facial motion among all frames. Then, the optical flow information are attained from the apex frame and a reference frame (i.e., onset frame). Finally, the optical flow features are fed into a pre-designed CNN model for further feature enhancement as well as to carry out the expression classification. To evaluate the effectiveness of OFF-ApexNet method, comprehensive evaluations are conducted on three public spontaneous micro-expression datasets (i.e., SMIC, CASME II and SAMM). The promising recognition result suggests that the proposed method can optimally describe the significant micro-expression details. In particular, we report that, in a multi-database with leave-one-subject-out cross-validation (LOSOCV) experimental protocol, the recognition performance reaches 74.60% of recognition accuracy and F-measure of 71.04%. We also note that this is the first work that performs cross-dataset validation on three databases in this domain.
103Â citations
Patent•
18 Apr 2003
TL;DR: In this paper, the authors proposed methods for processing tissue-derived spectral data for use in a tissue classification algorithm, which includes application of spectral and/or image masks for automatically separating ambiguous or unclassifiable spectral data from valid spectral data.
Abstract: The invention provides methods for processing tissue-derived spectral data for use in a tissue classification algorithm. Methods of the invention comprise application of spectral and/or image masks for automatically separating ambiguous or unclassifiable spectral data from valid spectral data. The invention improves the accuracy of tissue classification, in part, by properly identifying and accounting for spectral data from tissue regions that are affected by an obstruction and/or regions that lie outside a diagnostic zone of interest.
86Â citations
References
More filters
Book•
[...]
TL;DR: Robot Vision as discussed by the authors is a broad overview of the field of computer vision, using a consistent notation based on a detailed understanding of the image formation process, which can provide a useful and current reference for professionals working in the fields of machine vision, image processing, and pattern recognition.
Abstract: From the Publisher:
This book presents a coherent approach to the fast-moving field of computer vision, using a consistent notation based on a detailed understanding of the image formation process. It covers even the most recent research and will provide a useful and current reference for professionals working in the fields of machine vision, image processing, and pattern recognition.
An outgrowth of the author's course at MIT, Robot Vision presents a solid framework for understanding existing work and planning future research. Its coverage includes a great deal of material that is important to engineers applying machine vision methods in the real world. The chapters on binary image processing, for example, help explain and suggest how to improve the many commercial devices now available. And the material on photometric stereo and the extended Gaussian image points the way to what may be the next thrust in commercialization of the results in this area.
Chapters in the first part of the book emphasize the development of simple symbolic descriptions from images, while the remaining chapters deal with methods that exploit these descriptions. The final chapter offers a detailed description of how to integrate a vision system into an overall robotics system, in this case one designed to pick parts out of a bin.
The many exercises complement and extend the material in the text, and an extensive bibliography will serve as a useful guide to current research.
Errata (164k PDF)
3,783Â citations
TL;DR: In this article, a volume-rendering technique for the display of surfaces from sampled scalar functions of 3D spatial dimensions is discussed, which is not necessary to fit geometric primitives to the sampled data; images are formed by directly shading each sample and projecting it onto the picture plane.
Abstract: The application of volume-rendering techniques to the display of surfaces from sampled scalar functions of three spatial dimensions is discussed. It is not necessary to fit geometric primitives to the sampled data; images are formed by directly shading each sample and projecting it onto the picture plane. Surface-shading calculations are performed at every voxel with local gradient vectors serving as surface normals. In a separate step, surface classification operators are applied to compute a partial opacity of every voxel. Operators that detect isovalue contour surfaces and region boundary surfaces are examined. The technique is simple and fast, yet displays surfaces exhibiting smooth silhouettes and few other aliasing artifacts. The use of selective blurring and supersampling to further improve image quality is described. Examples from molecular graphics and medical imaging are given. >
2,437Â citations
Proceedings Article•
[...]
TL;DR: A scheme is developed for classifying the types of motion perceived by a humanlike robot and equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented.
Abstract: A scheme is developed for classifying the types of motion perceived by a humanlike robot. It is assumed that the robot receives visual images of the scene using a perspective system model. Equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented. >
2,000Â citations
"Three-dimensional surface reconstru..." refers background in this paper
...The equation only constrains the values of and to lie on a line in the gradient space....
[...]
TL;DR: A method that can handle sets of contours in which adjacent contours share a very contorted boundary is presented, and a new approach to solving the correspondence problem using a Minimum Spanning Tree generated from the contours is described.
Abstract: This paper is concerned with the problem of reconstructing the surfaces of three-dimensional objects, given a collection of planar contours representing cross-sections through the objects. This problem has important aplications in biomedical research and instruction, solid modeling, and industrial inspection.The method we describe produces a triangulated mesh from the data points of the contours which is then used in conjunction with a piecewise parametric surface-fitting algorithm to produce a reconstructed surface.The problem can be broken into four subproblems: the correspondence problem (which contours should be connected by the surface?), the tiling problem (how should the contours be connected?), the branching problem (what do we do when there are branches in the surface?), and the surface-fitting problem (what is the precise geometry of the reconstructed surface?) We describe our system for surface reconstruction from sets of contours with respect to each of these subproblems. Special attention is given to the correspondence and branching problems. We present a method that can handle sets of contours in which adjacent contours share a very contorted boundary, and we describe a new approach to solving the correspondence problem using a Minimum Spanning Tree generated from the contours.
345Â citations
"Three-dimensional surface reconstru..." refers background in this paper
...Much work has been done in the areas of shape from shading [1], shape from surface contour [2], and shape from texture gradient [3]....
[...]
TL;DR: An investigation of the quality of surface-shading algorithms using computer-simulated test objects is reported, and it was found that gray-level gradient shading and marching cubes did not differ greatly, except for thin objects, where adaptive gray- level gradient shading was better.
Abstract: An investigation of the quality of surface-shading algorithms using computer-simulated test objects is reported. Test results are presented for four algorithms: z-buffer gradient, gray-level gradient, adaptive gray-level gradient, and marching cubes with two extensions. It was found that gray-level gradient shading and marching cubes did not differ greatly, except for thin objects, where adaptive gray-level gradient shading was better. Transparent visualization using transparent gray-level gradient shading is examined. It is shown that although there is no way to assure the fidelity of transparent shading, it is useful when no other surface at all can be determined. Use of a combination of shading methods appears to yield the best visualization of the respective objects. >
272Â citations