Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

/pdf/deep-residual-learning-for-image-recognition-b75jg61qrq.pdf

Deep Residual Learning for Image Recognition

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. 
The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

In the realm of flexible automation, viewpoint independent object recognition will become increasingly important. An algorithm for the recognition of plane shapes under pseudo- perspective projection is presented. The method is based on the comparison of features that remain invariant under 2-D affine transformations. In particular, the work presented is part of the wider framework of semi-differential invariants. It was successfully tested on a database containing 42 objects.

Semidifferential invariants: algebraic-differential hybrids

https://lirias.kuleuven.be/bitstream/123456789/560177/1/4146_postprint.pdf

Dynamic filter networks for predicting unobserved views

Also recently, exciting strides forward have been made in the area of image restoration, particularly for image denoising and single image super-resolution. Deep learning techniques contributed to this significantly. The top methods differ in their formulations and assumptions, so even if their average performance may be similar, some work better on certain image types and image regions than others. This complementarity motivated us to propose a novel 3D convolutional fusion (3DCF) method. Unlike other methods adapted to different tasks, our method uses the exact same convolutional network architecture to address both image denoising and single image super-resolution. Our 3DCF method achieves substantial improvements (0.1 dB–0.4 dB PSNR) over the state-of-the-art methods that it fuses on standard benchmarks for both tasks. At the same time, the method still is computationally efficient.

Generic 3D Convolutional Fusion for Image Restoration

Geometry-driven evolution and functional optimization for curves

Being able to insert new objects into images is an important problem for both artistic image editing and for data augmentation. For a successful image manipulation, the plausible placement and the blending of the new objects in the image are critical. In this paper, we propose a fast method for the automatic selection of plausible locations for object insertion into images. Like previous work, we approach the object placement problem as a detection problem – given a bounding box, we evaluate whether an object is present inside the box based only on the neighborhood of the box. However, previous work requires a forward pass for each potential bounding box location. We propose instead to make use of masked convolutions to compute featuremaps for left, right, top and bottom contexts just once per image. Combining these features in such a way that no information from inside a bounding box is propagated to the final classifier allows the model to evaluate a grid of proposals on the featuremaps rather than on the image, speeding up inference dramatically. We validate that our model can generate plausible placements using experiments on the COCO dataset and on a user study. Our method trades off speed for performance, as compared to a patch based approach.

Luc Van Gool

Papers

Semidifferential invariants: algebraic-differential hybrids

Dynamic filter networks for predicting unobserved views

Generic 3D Convolutional Fusion for Image Restoration

Geometry-driven evolution and functional optimization for curves

Efficiently Detecting Plausible Locations for Object Placement Using Masked Convolutions