https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First , we highlight convolution with upsampled filters, or ‘atrous convolution’, as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second , we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third , we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed “DeepLab” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7 percent mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs. However, for many tasks, paired training data will not be available. We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. Our goal is to learn a mapping G : X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss. Because this mapping is highly under-constrained, we couple it with an inverse mapping F : Y → X and introduce a cycle consistency loss to push F(G(X)) ≈ X (and vice versa). Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc. Quantitative comparisons against several prior methods demonstrate the superiority of our approach.

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or 'atrous convolution', as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed "DeepLab" system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7% mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

Neural networks are usually trained by some form of stochastic gradient descent (SGD)). A number of strategies are in common use intended to improve SGD optimization, such as learning rate schedules, momentum, and batching. These are motivated by ideas about the occurrence of local minima at different scales, valleys, and other phenomena in the objective function. Empirical results presented here suggest that these phenomena are not significant factors in SGD optimization of MLP-related objective functions, and that the behavior of stochastic gradient descent in these problems is better described as the simultaneous convergence at different rates of many, largely non-interacting subproblems

On the Convergence of SGD Training of Neural Networks

In document image recognition, orientation detection of the scanned page is necessary for the following procedures to work correctly as they assume that the text is well oriented. Several methods have been proposed, but most of them rely on heuristics of the script such as the graphical asymmetry between ascenders and descenders for Roman script. The literature shows that as soon as this assumption is not fulfilled, e.g. plain capital text, noisy or degraded characters, etc. they fail. For a large-scale digitalization process, a low error and rejection rate are expected in order to reduce the amount of human intervention. We propose a Recognition Driven Page Orientation Detection (RD-POD) which does not depend on external criteria or assumption on the shape of the script. It uses the OCR engine for estimating the right orientation with a few lines of the document image. The RD-POD is highly robust and accurate, and is able to detect multiple orientations. Experimental evaluation shows that our method outperforms the current state-of-the-art on UW-1 dataset with an accuracy of 99.7%. Further tests on other three large and public datasets (MARG, ICDAR07, Google 1000 books) show accuracies of above 99% on each of them.

https://www.dfki.de/web/forschung/publikationen/renameFileForDownload?filename=2009-IUPR-21Aug_1307.pdf&file_id=uploads_354

Recognition Driven Page Orientation Detection

Kernel density estimators are established tools in nonparametric statistics. Due to their flexibility and ease of use, these methods are popular in computer vision and pattern recognition for tasks such as object tracking in video or image segmentation. The most frequently used algorithm for finding the modes in such densities (the mean shift) is a gradient ascent rule, which converges to local maxima. We propose a novel, globally optimal branch and bound algorithm for finding the modes in kernel densities. We show in experiments on datasets up to dimension five that the branch and bound method is faster than local optimization and observe linear runtime scaling of our method with sample size. Quantitative experiments on simulated data show that our method gives statistically significantly more accurate solutions than the mean shift. The mode localization accuracy is about five times more precise than that of the mean shift for all tested parameters. Applications to color image segmentation on an established benchmark test set also show measurably improved results when using global optimization.

https://www.dfki.de/web/forschung/publikationen/renameFileForDownload?filename=Branch&BoundWirjadiBreuel.pdf&file_id=uploads_363

A branch and bound algorithm for finding the modes in kernel density estimates

Abstract Although deep learning-based methods have dominated stereo matching leaderboards by yielding unprecedented disparity accuracy, their inference time is typically slow, i.e. , less than 4 FPS for a pair of 540p images. The main reason is that the leading methods employ time-consuming 3D convolutions applied to a 4D feature volume. A common way to speed up the computation is to downsample the feature volume, but this loses high-frequency details. To overcome these challenges, we propose a displacement-invariant cost computation module to compute the matching costs without needing a 4D feature volume. Rather, costs are computed by applying the same 2D convolution network on each disparity-shifted feature map pair independently. Unlike previous 2D convolution-based methods that simply perform context mapping between inputs and disparity maps, our proposed approach learns to match features between the two images. We also propose an entropy-based refinement strategy to refine the computed disparity map, which further improves the speed by avoiding the need to compute a second disparity map on the right image. Extensive experiments on standard datasets (SceneFlow, KITTI, ETH3D, and Middlebury) demonstrate that our method achieves competitive accuracy with much less inference time. On typical image sizes ( e.g. , $$540\times 960$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>540</mml:mn> <mml:mo>×</mml:mo> <mml:mn>960</mml:mn> </mml:mrow> </mml:math> ), our method processes over 100 FPS on a desktop GPU, making our method suitable for time-critical applications such as autonomous driving. We also show that our approach generalizes well to unseen datasets, outperforming 4D-volumetric methods. We will release the source code to ensure the reproducibility. 

/pdf/displacement-invariant-cost-computation-for-stereo-matching-moovwqwm.pdf

Displacement-Invariant Cost Computation for Stereo Matching

Semantic Indexing Task (SIN) Run No. Run ID Run Description infMAP (%) 1 F A IUPR-DFKI 1 Fisher Kernel + SVMs 2.86 2 F A IUPR-DFKI 2 Color Correlogram + SVMs 5.38 3 F A IUPR-DFKI 3 Fisher Kernel fused with Color Correlograms + SVMs 5.0 4 F A IUPR-DFKI 4 Fisher Kernel + kNN 0.71 Content-based Copy Detection (CCD) Run No. Run ID Run Description Opt.NDCR 1 *iupr-dfki.fsift F-SIFT+BoW+HE+EWGC 0.776 2 *iupr-dfki.fsift2 F-SIFT+BoW+HE+EWGC 0.923 3 SIFT SIFT+BoW+HE+EWGC 0.884 4 SIFT+PV SIFT+BoW+HE+EWGC+PV 0.501 5 F-SIFT+PV F-SIFT+BoW+HE+EWGC+PV 0.446 *: officially submitted run. This paper describes the TRECVID 2011 participation of the IUPR-DFKI team in the semantic indexing task (SIN) and content based copy detection task (CCD) task. For SIN, this years participation was dominated by an significant increase of vocabulary concept size from 130 to 346 concepts. In particular the system setup has been changed to last year’s participation [6] with respect to computational demands employing less computational costly features for classification and no usage of external training sources like YouTube. For CCD, this years participation is aimed at testing the flip invariant SIFT applied in video-only CCD. At the same time, we investigated how well we could achieve by relying on one keypoint feature alone.

/pdf/participation-at-trecvid-2011-semantic-indexing-content-2mvpdlz7ab.pdf

Thomas M. Breuel

Papers

On the Convergence of SGD Training of Neural Networks

Recognition Driven Page Orientation Detection

A branch and bound algorithm for finding the modes in kernel density estimates

Displacement-Invariant Cost Computation for Stereo Matching

Participation at TRECVID 2011 Semantic Indexing & Content-based Copy Detection Tasks.