Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

/pdf/histograms-of-oriented-gradients-for-human-detection-3nteinziuq.pdf

Histograms of oriented gradients for human detection

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

/pdf/rich-feature-hierarchies-for-accurate-object-detection-and-22fvjlggyg.pdf

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Image-based tracking control of an aerial blimp is proposed for surveillance systems. The controller is designed by backstepping techniques for underactuated systems. In our framework, only one camera on a blimp is used to control the blimp without the other sensors and to generate a reference trajectory from image information. After a commander sets a target, the blimp flies around it automatically. This image-based approach reduces to utilize the values estimated by a structure-from-motion algorithm. In this research an aerial blimp is used for indoor experiments, and image-based control are compared with position-based control.

Image-based tracking control of a blimp

The 3D Mosaic system is a vision system that incrementally reconstructs complex 3D scenes from multiple images. The system encompasses several levels of the vision process, starting with images and ending with symbolic scene descriptions. This paper describes the various components of the system, including stereo analysis, monocular analysis, and constructing and modifying the scene model In addition, the representation of the scene model is described. This model is intended for tasks such as matching, display generation, planning paths through the scene, and making other decisions about the scene environment. Examples showing how the system is used to interpret complex aerial photographs of urban scenes are presented. Each view of the scene, which may be either a single image or a stereo pair, undergoes analysis which results in a 3D wire-frame description that represents portions of edges and vertices of objects. The model is a surface-based description constructed from the wire frames. With each successive view, the model is incrementally updated and gradually becomes more accurate and complete. Task-specific knowledge, involving block-shaped objects in an urban scene, is used to extract the wire frames and construct and update the model.

/pdf/the-3d-mosaic-scene-understanding-system-incremental-1zsuokugjf.pdf

The 3D MOSAIC scene understanding system: incremental reconstruction of 3D scenes for complex images

The parameters estimated by Structure from Motion (SFM) contain inherent indeterminacies which we call gauge freedoms. Under a perspective camera, shape and motion parameters are only recovered up to an unknown similarity transformation. In this paper we investigate how covariance-based uncertainty can be represented under these gauge freedoms. Past work on uncertainty modeling has implicitly imposed gauge constraints on the solution before considering covariance estimation. Here we examine the effect of selecting a particular gauge on the uncertainty of parameters. We show potentially dramatic effects of gauge choice on parameter uncertainties. However the inherent geometric uncertainty remains the same irrespective of gauge choice. We derive a Geometric Equivalence Relationship with which covariances under different parametrizations and gauges can be compared, based on their true geometric uncertainty. We show that the uncertainty of gauge invariants exactly captures the geometric uncertainty of the solution, and hence provides useful measures for evaluating the uncertainty of the solution. Finally we propose a fast method for covariance estimation and show its correctness using the Geometric Equivalence Relationship.

Uncertainty Modeling for Optimal Structure from Motion

This paper describes a vision system that recognizes moving targets such as vehicles and pedestrians on public streets. This system can: (1) classify targets {vehicle, pedestrian, others} and, for "vehicles," discriminate vehicle types and (2) estimate the main colors of targets. According to the input images to the system, the categories of targets are set as {mule (golf cart for workers), sedan, van, truck, pedestrian (single or plural), and other (such as noise)}. Their colors are set as six color groups {red, orange, yellow; green; blue, light blue; white, silver, gray; dark blue, dark gray, black; dark red, dark orange}. In this experiment, we collected images of targets from 9: 00 a.m. to 5: 00 p.m. on sunny and cloudy days as system training samples. The recognition ratio was 91.1% under the condition that both the recognition results of type and color agreed with the operator's judgment. In addition, the system can detect predefined specific targets such as delivery vans, post office vans, and police cars by combining recognition results for type and color. The recognition ratio for specific targets was 92.9%. For the classification and estimation of targets, we employed a statistical linear discrimination method (linear discriminant analysis, LDA) and a nonlinear decision rule (weighted K-nearest neighbor rule, K-NN).

Type classification, color estimation, and specific target detection of moving targets on public streets

: The authors developed a noncontact proximity sensor which can measure the distance and orientation of a surface in a range of four to five centimeters. The sensor is based on the scheme of active illumination and triangulation. It uses multiple infrared LEDs(light emitting diodes) as the light sources and a PIN-diode area sensor chip for detecting the spot positions. Six LEDs with optics for collimating the beam are mounted at the sensor head. The directions of the beams are aligned to form a cone of light converging at a distance of 4.5 cm from the sensor head. As each LED is sequentially pulsed, the sensor chip detects the position, in its field of view, of the spot projected by the LED light beam on the object surface. The 3-D location of the spot on the surface can be computed by triangulation. By doing this for six LEDs a set of six 3-D points is obtained. Then by fitting a plane to those points, the distance and orientation of a small portion of the object surface are calculated. Since there is no moving part and the spot position sensor chip is a analog sensor which outputs the position of the spot directly without scanning its field of view, fast operation of the proximity sensor can be realized. Currently the sensor can give approximately 1000 measurements of distance and orientation per second with precision of 0.07 mm for distance and 1.5 deg for surface orientation. This non-contact proximity sensor will be useful for such applications as tracing an object surface by a robot arm with specified distance and orientation relative to the surface. (Author)

/pdf/an-optical-proximity-sensor-for-measuring-surface-position-1fid07r3zv.pdf

Takeo Kanade

Papers

Image-based tracking control of a blimp

The 3D MOSAIC scene understanding system: incremental reconstruction of 3D scenes for complex images

Uncertainty Modeling for Optimal Structure from Motion

Type classification, color estimation, and specific target detection of moving targets on public streets

An Optical Proximity Sensor for Measuring Surface Position and Orientation for Robot Manipulation.