Deep convolutional neural networks have performed remarkably well on many Computer Vision tasks. However, these networks are heavily reliant on big data to avoid overfitting. Overfitting refers to the phenomenon when a network learns a function with very high variance such as to perfectly model the training data. Unfortunately, many application domains do not have access to big data, such as medical image analysis. This survey focuses on Data Augmentation, a data-space solution to the problem of limited data. Data Augmentation encompasses a suite of techniques that enhance the size and quality of training datasets such that better Deep Learning models can be built using them. The image augmentation algorithms discussed in this survey include geometric transformations, color space augmentations, kernel filters, mixing images, random erasing, feature space augmentation, adversarial training, generative adversarial networks, neural style transfer, and meta-learning. The application of augmentation methods based on GANs are heavily covered in this survey. In addition to augmentation techniques, this paper will briefly discuss other characteristics of Data Augmentation such as test-time augmentation, resolution impact, final dataset size, and curriculum learning. This survey will present existing methods for Data Augmentation, promising developments, and meta-level decisions for implementing Data Augmentation. Readers will understand how Data Augmentation can improve the performance of their models and expand limited datasets to take advantage of the capabilities of big data.

/pdf/a-survey-on-image-data-augmentation-for-deep-learning-4ru1bj2hbb.pdf

A survey on Image Data Augmentation for Deep Learning

A van der Waals heterostructure is a type of metamaterial that consists of vertically stacked two-dimensional building blocks held together by the van der Waals forces between the layers. This design means that the properties of van der Waals heterostructures can be engineered precisely, even more so than those of two-dimensional materials. One such property is the 'twist' angle between different layers in the heterostructure. This angle has a crucial role in the electronic properties of van der Waals heterostructures, but does not have a direct analogue in other types of heterostructure, such as semiconductors grown using molecular beam epitaxy. For small twist angles, the moire pattern that is produced by the lattice misorientation between the two-dimensional layers creates long-range modulation of the stacking order. So far, studies of the effects of the twist angle in van der Waals heterostructures have concentrated mostly on heterostructures consisting of monolayer graphene on top of hexagonal boron nitride, which exhibit relatively weak interlayer interaction owing to the large bandgap in hexagonal boron nitride. Here we study a heterostructure consisting of bilayer graphene, in which the two graphene layers are twisted relative to each other by a certain angle. We show experimentally that, as predicted theoretically, when this angle is close to the 'magic' angle the electronic band structure near zero Fermi energy becomes flat, owing to strong interlayer coupling. These flat bands exhibit insulating states at half-filling, which are not expected in the absence of correlations between electrons. We show that these correlated states at half-filling are consistent with Mott-like insulator states, which can arise from electrons being localized in the superlattice that is induced by the moire pattern. These properties of magic-angle-twisted bilayer graphene heterostructures suggest that these materials could be used to study other exotic many-body quantum phases in two dimensions in the absence of a magnetic field. The accessibility of the flat bands through electrical tunability and the bandwidth tunability through the twist angle could pave the way towards more exotic correlated systems, such as unconventional superconductors and quantum spin liquids.

Correlated insulator behaviour at half-filling in magic-angle graphene superlattices

Correlated Insulator Behaviour at Half-Filling in Magic Angle Graphene Superlattices

Here we present deep-learning techniques for healthcare, centering our discussion on deep learning in computer vision, natural language processing, reinforcement learning, and generalized methods. We describe how these computational techniques can impact a few key areas of medicine and explore how to build end-to-end systems. Our discussion of computer vision focuses largely on medical imaging, and we describe the application of natural language processing to domains such as electronic health record data. Similarly, reinforcement learning is discussed in the context of robotic-assisted surgery, and generalized deep-learning methods for genomics are reviewed.

A guide to deep learning in healthcare.

The Brush Foundation studies on human growth and development, begun in 1931 and terminated in 1942, have been intensively reviewed and studied by Dr Greulich and Miss Pyle in the formulation of this Radiographic Atlas of Skeletal Development of the Hand and Wrist Serial radiographs of from 2 to 20 hand-films made at successive examinations of each of 1000 boys and girls made up the radiographic material Standards were selected that were judged to be the most representative of the central tendency or anatomic mode of each chronologic age group from birth through 18 years

Radiographic Atlas of Skeletal Development of the Hand and Wrist

Data augmentation is a commonly used technique for increasing both the size and the diversity of labeled training sets by leveraging input transformations that preserve corresponding output labels. In computer vision, image augmentations have become a common implicit regularization technique to combat overfitting in deep learning models and are ubiquitously used to improve performance. While most deep learning frameworks implement basic image transformations, the list is typically limited to some variations of flipping, rotating, scaling, and cropping. Moreover, image processing speed varies in existing image augmentation libraries. We present Albumentations, a fast and flexible open source library for image augmentation with many various image transform operations available that is also an easy-to-use wrapper around other augmentation libraries. We discuss the design principles that drove the implementation of Albumentations and give an overview of the key features and distinct capabilities. Finally, we provide examples of image augmentations for different computer vision tasks and demonstrate that Albumentations is faster than other commonly used image augmentation tools on most image transform operations.

Albumentations: fast and flexible image augmentations

Pixel-wise image segmentation is demanding task in computer vision. Classical U-Net architectures composed of encoders and decoders are very popular for segmentation of medical images, satellite images etc. Typically, neural network initialized with weights from a network pre-trained on a large data set like ImageNet shows better performance than those trained from scratch on a small dataset. In some practical applications, particularly in medicine and traffic safety, the accuracy of the models is of utmost importance. In this paper, we demonstrate how the U-Net type architecture can be improved by the use of the pre-trained encoder. Our code and corresponding pre-trained weights are publicly available at this https URL. We compare three weight initialization schemes: LeCun uniform, the encoder with weights from VGG11 and full network trained on the Carvana dataset. This network architecture was a part of the winning solution (1st out of 735) in the Kaggle: Carvana Image Masking Challenge.

TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation

Data augmentation is a commonly used technique for increasing both the size and the diversity of labeled training sets by leveraging input transformations that preserve output labels In computer vision domain, image augmentations have become a common implicit regularization technique to combat overfitting in deep convolutional neural networks and are ubiquitously used to improve performance While most deep learning frameworks implement basic image transformations, the list is typically limited to some variations and combinations of flipping, rotating, scaling, and cropping Moreover, the image processing speed varies in existing tools for image augmentation We present Albumentations, a fast and flexible library for image augmentations with many various image transform operations available, that is also an easy-to-use wrapper around other augmentation libraries We provide examples of image augmentations for different computer vision tasks and show that Albumentations is faster than other commonly used image augmentation tools on the most of commonly used image transformations The source code for Albumentations is made publicly available online at this https URL

Semantic segmentation of robotic instruments is an important problem for the robot-assisted surgery. One of the main challenges is to correctly detect an instrument's position for the tracking and pose estimation in the vicinity of surgical scenes. Accurate pixel-wise instrument segmentation is needed to address this challenge. In this paper we describe our deep learning-based approach for robotic instrument segmentation. Our approach demonstrates an improvement over the state-of-the-art results using several deep neural network architectures. It addressed the binary segmentation problem, where every pixel in an image is labeled as an instrument or background from the surgery video feed. In addition, we address a multi-class segmentation problem, in which we distinguish between different instruments or different parts of an instrument from the background. In this setting, our approach outperforms other methods for automatic instrument segmentation thereby providing state-of-the-art results for these problems. The source code for our solution is made publicly available at https://github.com/ternaus/robot-surgery-segmentation.

Automatic Instrument Segmentation in Robot-Assisted Surgery using Deep Learning

Breast cancer is one of the main causes of cancer death worldwide. Early diagnostics significantly increases the chances of correct treatment and survival, but this process is tedious and often leads to disagreement between pathologists. Computer-aided diagnosis systems show potential for improving the diagnostic accuracy. In this work, we develop the computational approach based on deep convolution neural networks for breast cancer histology image classification. Hematoxylin and eosin stained breast histology microscopy image dataset is provided as a part of the ICIAR 2018 Grand Challenge on Breast Cancer Histology Images. Our approach utilizes several deep neural network architectures and gradient boosted trees classifier. For 4-class classification task, we report 87.2% accuracy. For 2-class classification task to detect carcinomas we report 93.8% accuracy, AUC 97.3%, and sensitivity/specificity 96.5/88.0% at the high-sensitivity operating point. To our knowledge, this approach outperforms other common methods in automated histopathological image classification. The source code for our approach is made publicly available at https://github.com/alexander-rakhlin/ICIAR2018.

Vladimir Iglovikov

Papers

Albumentations: fast and flexible image augmentations

TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation

Albumentations: fast and flexible image augmentations

Automatic Instrument Segmentation in Robot-Assisted Surgery using Deep Learning

Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis