Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

/pdf/histograms-of-oriented-gradients-for-human-detection-3nteinziuq.pdf

Histograms of oriented gradients for human detection

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

/pdf/rich-feature-hierarchies-for-accurate-object-detection-and-22fvjlggyg.pdf

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Understanding the heterogeneous behavior of hematopoietic stem cells (HSCs) is required for the expansion of the cells without loss of their regenerative capacity. As such, it is essential to establish their lineage relationships by tracking the history of individual cells in a cell population. However, the quality of lineage relationships is often degraded because of undetected or misdetected mitotic events, which lead to missed or inaccurate mother-daughter cell relationships. In this paper, we present an automated mitosis detection method for HSCs in time-lapse phase-contrast microscopy images. Since HSCs are nonadherent, i.e., free-floating in the culture medium, the method is distinguished from the recent mitosis detection methods developed for adherent cells that are attached to the surface of a petri dish. The proposed mitosis detection method detects individual cells in each image frame and subsequently tracks them over time and in so doing identifies newly appeared cells, each of which is considered as a candidate of a newborn cell. Each candidate is then examined to determine whether it is indeed a newborn cell based on temporal change of cell sizes of potential mother and daughter cells. Our method was quantitatively evaluated on 14 HSC populations, each of which is observed for four days, resulting in a precision of 97.4% and a recall of 96.6%.

/pdf/mitosis-detection-of-hematopoietic-stem-cell-populations-in-1rta9rq9ng.pdf

Mitosis detection of hematopoietic stem cell populations in time-lapse phase-contrast microscopy images

We present a novel approach to segmenting video using iterated graph cuts based on spatio-temporal volumes. We use the mean shift clustering algorithm to build the spatio-temporal volumes with different bandwidths from the input video. We compute the prior probability obtained by the likelihood from a color histogram and a distance transform using the segmentation results from graph cuts in the previous process, and set the probability as the t-link of the graph for the next process. The proposed method can segment regions of an object with a stepwise process from global to local segmentation by iterating the graph-cuts process with mean shift clustering using a different bandwidth. It is possible to reduce the number of nodes and edges to about 1/25 compared to the conventional method with the same segmentation rate.

Video segmentation using iterated graph cuts based on spatio-temporal volumes

High-throughput automated analysis of cell population behaviors in vitro is of great importance to biological research. In particular, automated quantification of cellular mitosis in time-lapse microscopy video is useful for multiple applications such as tissue engineering, cancer research, and developmental biology. Accurate localization and counting of mitosis are challenging since cells undergo drastic morphological and appearance changes during mitosis. To tackle this challenge, we propose a fully-automated detection method for cells imaged with phase contrast microscopy. The method consists of three stages: image preconditioning, spatiotemporal volume extraction and SVM — based mitosis event detection. First, the input images are transformed based on physics of phase contrast image formation such that potential mitosis regions are assigned high values. Second, volumetric region grow was performed on the transformed images to extract candidate mitosis regions. Third, mitosis events are detected in the candidates using a Support Vector Machine (SVM) classifier. The proposed method does not depend on empirical parameters, ad hoc image processing, or explicit cell tracking; and can be straightforwardly adapted to different cell types. It was validated with 10 image sequences consisting of 8000 images, and achieved excellent performance with 90.6% average precision and 95.6% average recall.

Spatiotemporal mitosis event detection in time-lapse phase contrast microscopy image sequences

A three-dimensional perception system for building a geometrical description of rugged terrain environments from range image data is presented with reference to the exploration of the rugged terrain of Mars. An intermediate representation consisting of an elevation map that includes an explicit representation of uncertainty and labeling of the occluded regions is proposed. The locus method used to convert range image to an elevation map is introduced, along with an uncertainty model based on this algorithm. Both the elevation map and the locus method are the basis of a terrain matching algorithm which does not assume any correspondences between range images. The two-stage algorithm consists of a feature-based matching algorithm to compute an initial transform and an iconic terrain matching algorithm to merge multiple range images into a uniform representation. Terrain modeling results on real range images of rugged terrain are presented. The algorithms considered are a fundamental part of the perception system for the Ambler, a legged locomotor.

Perception For Rugged Terrain

The factorization method, first developed by Tomasi and Kanade (1992), recovers both the shape of an object and its motion from a sequence of images, using many images and tracking many feature points to obtain highly redundant feature position information The method robustly processes the feature trajectory information using singular value decomposition (SVD), taking advantage of the linear algebraic properties of orthographic projection However, an orthographic formulation limits the range of motions the method can accommodate Paraperspective projection, first introduced by Ohta et al (1981), is a projection model that closely approximates perspective projection by modeling several effects not modeled under orthographic projection, while retaining linear algebraic properties Our paraperspective factorization method can be applied to a much wider range of motion scenarios, including image sequences containing motion toward the camera and aerial image sequences of terrain taken from a low-altitude airplane

Takeo Kanade

Papers

Mitosis detection of hematopoietic stem cell populations in time-lapse phase-contrast microscopy images

Video segmentation using iterated graph cuts based on spatio-temporal volumes

Spatiotemporal mitosis event detection in time-lapse phase contrast microscopy image sequences

Perception For Rugged Terrain

A paraperspective factorization method for shape and motion recovery