Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

/pdf/histograms-of-oriented-gradients-for-human-detection-3nteinziuq.pdf

Histograms of oriented gradients for human detection

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

/pdf/rich-feature-hierarchies-for-accurate-object-detection-and-22fvjlggyg.pdf

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

An image processing system that may be used to detect changes in objects, such as damage to automobiles, compares a base object model, which depicts the object in an expected condition, to one or more target object images of the object in the changed condition. To do so, the image processing system first processes a target object image to detect one or more predefined landmarks in the target object image using one or more correlation filters. The image processing system then uses the detected landmarks to determine a camera model for the target object image and uses the camera model to correct for camera distortions and to align the target object depicted in a target object image with the object in the base object model to put these objects in a common frame of reference for use in subsequent image processing. The image processing system may then determine contours of the target object within the target object image by overlaying an aligned base object model with the target object image, may remove background pixels or other extraneous information based on this comparison, and may perform a statistical processing routine on the identified target object image to detect changes to the target object as depicted in the target object image as compared to the base object model.

Image processing system to align a target object in a target object image with an object model

The 3D Mosaic system is a vision system that incrementally reconstructs complex 3D scenes from a sequence of images obtained from multiple viewpoints. The system encompasses several levels of the vision process, starting with images and ending with symbolic scene descriptions. This paper describes the various components of the system, including stereo analysis, monocular analysis, and constructing and updating the scene model. In addition, the representation of the scene model is described. This model is intended for tasks such as matching, display generation, planning paths through the scene, and making other decisions about the scene environment. Examples showing how the system is used to interpret complex aerial photographs of urban scenes are presented. Each view of the scene, which may be either a single image or a stereo pair, undergoes analysis which results in a 3D wire-frame description that represents portions of edges and vertices of objects. The model is a surface-based description constructed from the wire frames. With each successive view, the model is incrementally updated and gradually becomes more accurate and complete. Task-specific knowledge, involving block-shaped objects in an urban scene, is used to extract the wire frames and construct and update the model. The model is represented as a graph in terms of symbolic primitives such as faces, edges, vertices, and their topology and geometry. This permits the representation of partially complete, planar-faced objects. Because incremental modifications to the model must be easy to perform, the model contains mechanisms to (1) add primitives in a manner such that constraints on geometry imposed by these additions are propagated throughout the model, and (2) modify and delete primitives if discrepancies arise between newly derived and current information. The model also contains mechanisms that permit the generation, addition, and deletion of hypotheses for parts of the scene for which there is little data.

/pdf/incremental-reconstruction-of-3d-scenes-from-multiple-5ge4sulwor.pdf

Incremental Reconstruction of 3D Scenes from Multiple Complex Images

This chapter focuses on image analysis and understanding of live cell populations in time lapse phase contrast microscopy. The computer vision tasks involve cell segmentation and cell behavior understanding, including cell migration, division (mitosis), death (apoptosis), and differentiation. We will describe the problem definition for each topic, introduce the general schools of approaches that have been explored, discuss details of the state-of-the-art algorithms, and propose promising directions for future investigation.

Detect Cells and Cellular Behaviors in Phase Contrast Microscopy Images

: The contract made significant progress across a broad front on the problems of computer vision for outdoor mobile robots New algorithms were built in neural networks, range data analysis, object recognition and road finding Perception modules were integrated into new systems (including on road and off road, notably on the new Navlab II vehicle); and there were notable programmatic events, ranging from generating two new thesis proposals to playing a major role in the 'Tiger Team', shaping the architecture for the new DARPA program in Unmanned Ground Vehicles This report begins with a summary of the year's activities and accomplishments, in this chapter Chapter 2, '3-D Landmark Recognition from Range Image', provides more detail on object recognition from multiple sensor locations Chapter 3, 'Representation and Recovery of Road Geometry in YARF', discusses geometry issues in YARF, our symbolic road tracking system The last two chapters discuss systems issues that are important in providing cues and constraints for an active vision approach to robot driving' A Computational Model of Driving for Autonomous Vehicles', Chapter 4, introduces the complexities of reasoning for driving in traffic The fifth and final chapter, 'Combining artificial neural networks and symbolic processing for autonomous robot guidance', shows how we combine neural nets with map data in a complete system

Perception for Outdoor Navigation

When addressing the problem of scene understanding from a single image, we want our system to understand not only where objects are in the image, but also where they are in the 3D world. Segmenting and labeling regions only in the 2D image plane does not achieve this goal. We need a representation that inherently encodes the 3D properties of the scene. In addition to understanding the location in 3D, we also want our system to make use of physical knowledge about valid configurations of our world by rejecting configurations that violate physical constraints, such as two objects occupying the same volume. 3D geometric properties can also aid in detecting and identifying certain classess of objects that are well characterized by their geometry. In this thesis, we will demonstrate the benefits of using 3D representation for indoor scene understanding. We will show that the use of models provides a natural way to represent objects in 3D and inject knowledge we have about the world to perform geometric reasoning.

Takeo Kanade

Papers

Image processing system to align a target object in a target object image with an object model

Incremental Reconstruction of 3D Scenes from Multiple Complex Images

Detect Cells and Cellular Behaviors in Phase Contrast Microscopy Images

Perception for Outdoor Navigation

Three dimensional representation and reasoning for indoor scene understanding