There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .

/pdf/u-net-convolutional-networks-for-biomedical-image-2xoyjzggxv.pdf

U-Net: Convolutional Networks for Biomedical Image Segmentation

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.

/pdf/microsoft-coco-common-objects-in-context-55ihxq1zt5.pdf

Microsoft COCO: Common Objects in Context

In this paper, we suggest to model priors on human motion by means of nonparametric kernel densities. Kernel densities avoid assumptions on the shape of the underlying distribution and let the data speak for themselves. In general, kernel density estimators suffer from the problem known as the curse of dimensionality, i.e., the amount of data required to cover the whole input space grows exponentially with the dimension of this space. In many applications, such as human motion tracking, though, this problem turns out to be less severe, since the relevant data concentrate in a much smaller subspace than the original high-dimensional space. As we demonstrate in this paper, the concentration of human motion data on lower-dimensional manifolds, approves kernel density estimation as a transparent tool that is able to model priors on arbitrary mixtures of human motions. Further, we propose to support the ability of kernel estimators to capture distributions on low-dimensional manifolds by replacing the standard isotropic kernel by an adaptive, anisotropic one.

Nonparametric density estimation with adaptive, anisotropic kernels for human motion tracking

We present a variational approach for surface reconstruction from a set of oriented points with scale information. We focus particularly on scenarios with non-uniform point densities due to images taken from different distances. In contrast to previous methods, we integrate the scale information in the objective and globally optimize the signed distance function of the surface on a balanced octree grid. We use a finite element discretization on the dual structure of the octree minimizing the number of variables. The tetrahedral mesh is generated efficiently from the dual structure, and also memory efficiency is optimized, such that robust data terms can be used even on very large scenes. The surface normals are explicitly optimized and used for surface extraction to improve the reconstruction at edges and corners.

/pdf/global-dense-multiscale-reconstruction-for-a-billion-points-1obmg1mrfy.pdf

Global, Dense Multiscale Reconstruction for a Billion Points

We train a generative convolutional neural network which is able to generate images of objects given object type, viewpoint, and color. We train the network in a supervised manner on a dataset of rendered 3D chair models. Our experiments show that the network does not merely learn all images by heart, but rather finds a meaningful representation of a 3D chair model allowing it to assess the similarity of different chairs, interpolate between given viewpoints to generate the missing ones, or invent new chair styles by interpolating between chairs from the training set. We show that the network can be used to find correspondences between different chairs from the dataset, outperforming existing approaches on this task.

Learning to Generate Chairs with Convolutional Neural Networks

Background: Measurement of muscle fiber size and determination of size distribution is important in the assessment of neuromuscular disease. Fiber size estimation by simple inspection is inaccurate and subjective. Manual segmentation and measurement are time-consuming and tedious. We therefore propose an automated image analysis method for objective, reproducible, and time-saving measurement of muscle fibers in routinely hematoxylineosin stained cryostat sections. Methods: The proposed segmentation technique makes use of recent advances in level set based segmentation, where classical edge based active contours are extended by region based cues, such as color and texture. Segmentation and measurement are performed fully automatically. Multiple morphometric parameters, i.e., cross sectional area, lesser diameter, and perimeter are assessed in a single pass. The performance of the computed method was compared to results obtained by manual measurement by experts. Results: The correct classification rate of the computed method was high (98%). Segmentation and measurement results obtained manually or automatically did not reveal any significant differences. Conclusions: The presented region based active contour approach has been proven to accurately segment and measure muscle fibers. Complete automation minimizes user interaction, thus, batch processing, as well as objective and reproducible muscle fiber morphometry are provided. q 2007 International Society for Analytical Cytology

/pdf/fully-automated-segmentation-and-morphometrical-analysis-of-ilrjiviko9.pdf

Fully automated segmentation and morphometrical analysis of muscle fiber images.

Deblurring with a spatially invariant kernel of arbitrary shape is a frequent problem in image processing. We address this task by studying nonconvex variational functionals that lead to diffusion-reaction equations of Perona–Malik type. Further we consider novel deblurring PDEs with anisotropic diffusion tensors. In order to improve deblurring quality we propose a continuation strategy in which the diffusion weight is reduced during the process. To evaluate our methods, we compare them to two established techniques: Wiener filtering which is regarded as the best linear filter, and a total variation based deconvolution which is the most widespread deblurring PDE. The experiments confirm the favourable performance of our methods, both visually and in terms of signal-to-noise ratio.

/pdf/pde-based-deconvolution-with-forward-backward-diffusivities-1ooxknnyan.pdf

Thomas Brox

Papers

Nonparametric density estimation with adaptive, anisotropic kernels for human motion tracking

Global, Dense Multiscale Reconstruction for a Billion Points

Learning to Generate Chairs with Convolutional Neural Networks

Fully automated segmentation and morphometrical analysis of muscle fiber images.

PDE-based deconvolution with forward-backward diffusivities and diffusion tensors