Showing papers by "Takeo Kanade published in 2012"

Reconstructing 3d human pose from 2d image landmarks

Abstract: Reconstructing an arbitrary configuration of 3D points from their projection in an image is an ill-posed problem. When the points hold semantic meaning, such as anatomical landmarks on a body, human observers can often infer a plausible 3D configuration, drawing on extensive visual memory. We present an activity-independent method to recover the 3D configuration of a human figure from 2D locations of anatomical landmarks in a single image, leveraging a large motion capture corpus as a proxy for visual memory. Our method solves for anthropometrically regular body pose and explicitly estimates the camera via a matching pursuit algorithm operating on the image projections. Anthropometric regularity (i.e., that limbs obey known proportions) is a highly informative prior, but directly applying such constraints is intractable. Instead, we enforce a necessary condition on the sum of squared limb-lengths that can be solved for in closed form to discourage implausible configurations in 3D. We evaluate performance on a wide variety of human poses captured from different viewpoints and show generalization to novel 3D configurations and robustness to missing data.

Real-time topometric localization

Abstract: Autonomous vehicles must be capable of localizing even in GPS denied situations. In this paper, we propose a real-time method to localize a vehicle along a route using visual imagery or range information. Our approach is an implementation of topometric localization, which combines the robustness of topological localization with the geometric accuracy of metric methods. We construct a map by navigating the route using a GPS-equipped vehicle and building a compact database of simple visual and 3D features. We then localize using a Bayesian filter to match sequences of visual or range measurements to the database. The algorithm is reliable across wide environmental changes, including lighting differences, seasonal variations, and occlusions, achieving an average localization accuracy of 1 m over an 8 km route. The method converges correctly even with wrong initial position estimates solving the kidnapped robot problem.

First-Person Vision

Abstract: For understanding the behavior, intent, and environment of a person, the surveillance metaphor is traditional; that is, install cameras and observe the subject, and his/her interaction with other people and the environment. Instead, we argue that the first-person vision (FPV), which senses the environment and the subject's activities from a wearable sensor, is more advantageous with images about the subject's environment as taken from his/her view points, and with readily available information about head motion and gaze through eye tracking. In this paper, we review key research challenges that need to be addressed to develop such FPV systems, and describe our ongoing work to address them using examples from our prototype systems.

A Semi-Markov Model for Mitosis Segmentation in Time-Lapse Phase Contrast Microscopy Image Sequences of Stem Cell Populations

Abstract: We propose a semi-Markov model trained in a max-margin learning framework for mitosis event segmentation in large-scale time-lapse phase contrast microscopy image sequences of stem cell populations. Our method consists of three steps. First, we apply a constrained optimization based microscopy image segmentation method that exploits phase contrast optics to extract candidate subsequences in the input image sequence that contains mitosis events. Then, we apply a max-margin hidden conditional random field (MM-HCRF) classifier learned from human-annotated mitotic and nonmitotic sequences to classify each candidate subsequence as a mitosis or not. Finally, a max-margin semi-Markov model (MM-SMM) trained on manually-segmented mitotic sequences is utilized to reinforce the mitosis classification results, and to further segment each mitosis into four predefined temporal stages. The proposed method outperforms the event-detection CRF model recently reported by Huh as well as several other competing methods in very challenging image sequences of multipolar-shaped C3H10T1/2 mesenchymal stem cells. For mitosis detection, an overall precision of 95.8% and a recall of 88.1% were achieved. For mitosis segmentation, the mean and standard deviation for the localization errors of the start and end points of all mitosis stages were well below 1 and 2 frames, respectively. In particular, an overall temporal location error of 0.73 ±1.29 frames was achieved for locating daughter cell birth events.

Maximum Entropy for Collaborative Filtering

Abstract: Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order interactions is generally not present. Second, the variables that we are conditioning upon vary for each query. That is, users label different variables during each query. For this reason, there is no consistent input to output mapping. To address these problems we purpose a maximum entropy approach using a non-standard measure of entropy. This approach can be simplified to solving a set of linear equations that can be efficiently solved.

6D pose estimation of textureless shiny objects using random ferns for bin-picking

Abstract: We address the problem of 6D pose estimation of a textureless and shiny object from single-view 2D images, for a bin-picking task. For a textureless object like a mechanical part, conventional visual feature matching usually fails due to the absence of rich texture features. Hierarchical template matching assumes that few templates can cover all object appearances. However, the appearance of a shiny object largely depends on its pose and illumination. Furthermore, in a bin-picking task, we must cope with partial occlusions, shadows, and inter-reflections.

Fast reactive control for illumination through rain and snow

Abstract: During low-light conditions, drivers rely mainly on headlights to improve visibility. But in the presence of rain and snow, headlights can paradoxically reduce visibility due to light reflected off of precipitation back towards the driver. Precipitation also scatters light across a wide range of angles that disrupts the vision of drivers in oncoming vehicles. In contrast to recent computer vision methods that digitally remove rain and snow streaks from captured images, we present a system that will directly improve driver visibility by controlling illumination in response to detected precipitation. The motion of precipitation is tracked and only the space around particles is illuminated using fast dynamic control. Using a physics-based simulator, we show how such a system would perform under a variety of weather conditions. We build and evaluate a proof-of-concept system that can avoid water drops generated in the laboratory.

Apoptosis detection for adherent cell populations in time-lapse phase-contrast microscopy images

Abstract: The detection of apoptosis, or programmed cell death, is important to understand the underlying mechanism of cell development. At present, apoptosis detection resorts to fluorescence or colorimetric assays, which may affect cell behavior and thus not allow long-term monitoring of intact cells. In this work, we present an image analysis method to detect apoptosis in time-lapse phase-contrast microscopy, which is non-destructive imaging. The method first detects candidates for apoptotic cells based on the optical principle of phase-contrast microscopy in connection with the properties of apoptotic cells. The temporal behavior of each candidate is then examined in its neighboring frames in order to determine if the candidate is indeed an apoptotic cell. When applied to three C2C12 myoblastic stem cell populations, which contain more than 1000 apoptosis, the method achieved around 90% accuracy in terms of average precision and recall.

Phase Contrast Image Restoration Via Dictionary Representation of Diffraction Patterns

Abstract: The restoration of microscopy images makes the segmentation and detection of cells easier and more reliable, which facilitates automated cell tracking and cell behavior analysis. In this paper, the authors analyze the image formation process of phase contrast images and propose an image restoration method based on the dictionary representation of diffraction patterns. By formulating and solving a min-l1 optimization problem, each pixel is restored into a feature vector corresponding to the dictionary representation. Cells in the images are then segmented by the feature vector clustering. In addition to segmentation, since the feature vectors capture the information on the phase retardation caused by cells, they can be used for cell stage classification between intermitotic and mitotic/apoptotic stages. Experiments on three image sequences demonstrate that the dictionary-based restoration method can restore phase contrast images containing cells with different optical natures and provide promising results on cell stage classification.

Multi-label classification for the analysis of human motion quality

Abstract: Knowing how well an activity is performed is important for home rehabilitation. We would like to not only know if a motion is being performed correctly, but also in what way the motion is incorrect so that we may provide feedback to the user. This paper describes methods for assessing human motion quality using body-worn tri-axial accelerometers and gyroscopes. We use multi-label classifiers to detect subtle errors in exercise performances of eight individuals with knee osteoarthritis, a degenerative disease of the cartilage. We present results obtained using various machine learning methods with decision tree base classifiers. The classifier can detect classes in multi-label data with 75% sensitivity, 90% specificity and 80% accuracy. The methods presented here form the basis for an at-home rehabilitation device that will recognize errors in patient exercise performance, provide appropriate feedback on the performance, and motivate the patient to continue the prescribed regimen.

Connecting missing links: object discovery from sparse observations using 5 million product images

Abstract: Object discovery algorithms group together image regions that originate from the same object. This process is effective when the input collection of images contains a large number of densely sampled views of each object, thereby creating strong connections between nearby views. However, existing approaches are less effective when the input data only provide sparse coverage of object views. We propose an approach for object discovery that addresses this problem. We collect a database of about 5 million product images that capture 1.2 million objects from multiple views. We represent each region in the input image by a "bag" of database object regions. We group input regions together if they share similar "bags of regions." Our approach can correctly discover links between regions of the same object even if they are captured from dramatically different viewpoints. With the help from these added links, our proposed approach can robustly discover object instances even with sparse coverage of the viewpoints.

Automatic acquisition of a 3D eye model for a wearable first-person vision device

Abstract: A wearable gaze tracking device can work with users in daily-life. For long time of use, a non-active method that does not employ an infrared illumination system is desirable from safety standpoint. It is well known that the eye model constraints substantially improve the accuracy and robustness of gaze estimation. However, the eye model needs to be calibrated for each person and each device. We propose a method to automatically build the eye model for a wearable gaze tracking device. The key idea is that the eye model, which includes the eye structure and eye-camera relationship, impose constraints on image analysis even when it is incomplete, so we adopt an iterative eye model building process with gradually increasing eye model constraints. Performance of the proposed method is evaluated in various situations, including different eye colors of users and camera configurations. We have confirmed that the gaze tracking system using our eye model works well under general situations: indoor, outdoor and driving scene.

Quality of Life Technology [Scanning the Issue]

Abstract: The articles in this special issue focus on advancements in quality of life technology, or QoLT. This is defined as intelligent systems that augment body and mind functions for self-determination for older adults and people with disabilities. QoLT systems can take many forms: they could be a device that a person carries or wears, a mobile system that accompanies a person, or a technology-embedded environment in which a person lives.

Mitosis detection of hematopoietic stem cell populations in time-lapse phase-contrast microscopy images

Abstract: Understanding the heterogeneous behavior of hematopoietic stem cells (HSCs) is required for the expansion of the cells without loss of their regenerative capacity. As such, it is essential to establish their lineage relationships by tracking the history of individual cells in a cell population. However, the quality of lineage relationships is often degraded because of undetected or misdetected mitotic events, which lead to missed or inaccurate mother-daughter cell relationships. In this paper, we present an automated mitosis detection method for HSCs in time-lapse phase-contrast microscopy images. Since HSCs are nonadherent, i.e., free-floating in the culture medium, the method is distinguished from the recent mitosis detection methods developed for adherent cells that are attached to the surface of a petri dish. The proposed mitosis detection method detects individual cells in each image frame and subsequently tracks them over time and in so doing identifies newly appeared cells, each of which is considered as a candidate of a newborn cell. Each candidate is then examined to determine whether it is indeed a newborn cell based on temporal change of cell sizes of potential mother and daughter cells. Our method was quantitatively evaluated on 14 HSC populations, each of which is observed for four days, resulting in a precision of 97.4% and a recall of 96.6%.

Capturability analysis of a three-dimensional guidance law with angular acceleration input

Abstract: This paper presents the capturability analysis of a three-dimensional guidance law that provides a desired angular acceleration of the heading angle to a pursuer. The relative rotational motion between a pursuer and an evader is represented by an equation similar to the equation of motion for a spherical pendulum with a disturbance. A set of sufficient conditions for successful pursuit are derived, and it is proved that the conditions are satisfied by suitably choosing the gain parameters of the guidance law. For sufficiently large gains, a pursuer controlled by the guidance law catches a maneuvering evader while achieving motion camouflage.

Approximate representation of unknown objects with a single-line scanning lidar and a video camera

Abstract: Models are useful for many computer vision tasks, such as object detection, recognition, and tracking. Computer vision tasks must handle situations where unknown objects appear and must detect and track some object which is not in the trained database. In such cases, the system must learn or, otherwise derive, descriptions of new objects. In this dissertation, we investigate creating a representation of previously unknown objects that newly appear in the scene. The representation is to create a viewpoint-invariant and scale-normalized model approximately describing an unknown object. Those properties of the representation facilitate 3D tracking of the object using 2D-to-2D image matching. The representation has both benefits of an implicit model (referred to as a view-based model) and an explicit model (referred to as a shape-based model). The object representation is created using multi-modal sensors. We illustrate the benefits of the object representation with two applications: object detection and 3D tracking. We extend the object representation to an explicit model by imposing a shape prior and combining two existing approaches.