During the past decade, deep learning is one of the essential breakthroughs made in artificial intelligence. In particular, it has achieved great success in image processing. Correspondingly, various applications related to image processing are also promoting the rapid development of deep learning in all aspects of network structure, layer designing, and training tricks. However, the deeper structure makes the back-propagation algorithm more difficult. At the same time, the scale of training images without labels is also rapidly increasing, and class imbalance severely affects the performance of deep learning, these urgently require more novelty deep models and new parallel computing system to more effectively interpret the content of the image and form a suitable analysis mechanism. In this context, this survey provides four deep learning model series, which includes CNN series, GAN series, ELM-RVFL series, and other series, for comprehensive understanding towards the analytical techniques of image processing field, clarify the most important advancements and shed some light on future studies. By further studying the relationship between deep learning and image processing tasks, which can not only help us understand the reasons for the success of deep learning but also inspires new deep models and training methods. More importantly, this survey aims to improve or arouse other researchers to catch a glimpse of the state-of-the-art deep learning methods in the field of image processing and facilitate the applications of these deep learning technologies in their research tasks. Besides, we discuss the open issues and the promising directions of future research in image processing using the new generation of deep learning.

/pdf/a-survey-on-the-new-generation-of-deep-learning-in-image-25ttc5qdex.pdf

A Survey on the New Generation of Deep Learning in Image Processing

Using state-of-the-art deep learning models for cancer diagnosis presents several challenges related to the nature and availability of labeled histology images. In particular, cancer grading and localization in these images normally relies on both image- and pixel-level labels, the latter requiring a costly annotation process. In this survey, deep weakly-supervised learning (WSL) models are investigated to identify and locate diseases in histology images, without the need for pixel-level annotations. Given training data with global image-level labels, these models allow to simultaneously classify histology images and yield pixel-wise localization scores, thereby identifying the corresponding regions of interest (ROI). Since relevant WSL models have mainly been investigated within the computer vision community, and validated on natural scene images, we assess the extent to which they apply to histology images which have challenging properties, e.g. very large size, similarity between foreground/background, highly unstructured regions, stain heterogeneity, and noisy/ambiguous labels. The most relevant models for deep WSL are compared experimentally in terms of accuracy (classification and pixel-wise localization) on several public benchmark histology datasets for breast and colon cancer -- BACH ICIAR 2018, BreaKHis, CAMELYON16, and GlaS. Furthermore, for large-scale evaluation of WSL models on histology images, we propose a protocol to construct WSL datasets from Whole Slide Imaging. Results indicate that several deep learning models can provide a high level of classification accuracy, although accurate pixel-wise localization of cancer regions remains an issue for such images. Code is publicly available.

Deep weakly-supervised learning methods for classification and localization in histology images: a survey.

A novel ensemble local graph structure based feature extraction network for EEG signal analysis

Content-based image retrieval (CBIR) is a widely used technique for retrieval images from huge and unlabeled image databases. However, users are not satisfied with the traditional information retrieval techniques. Moreover, the emergence of web development and transmission networks and also the amount of images which are available to users continue to grow. Therefore, a permanent and considerable digital image production in many areas takes place. Hence, the rapid access to these huge collections of images and retrieve similar image of a given image (Query) from this large collection of images presents major challenges and requires efficient techniques. The performance of a content-based image retrieval system crucially depends on the feature representation and similarity measurement. For this reason, we present, on this paper, a simple but effective deep learning framework based on Convolutional Neural Networks (CNN) and Support Vector Machine (SVM) for fast image retrieval composed of feature extraction and classification. From several extensive of empirical studies for a variety of CBIR tasks using image database, we obtain some encouraging results which reveals several important insights for improving the CBIR performance.

Content-Based Image Retrieval Using Convolutional Neural Networks

Human pose estimation is a well-known problem in computer vision to locate joint positions Existing datasets for learning of poses are observed to be not challenging enough in terms of pose diversity, object occlusion and view points This makes the pose annotation process relatively simple and restricts the application of the models that have been trained on them To handle more variety in human poses, we propose the concept of fine-grained hierarchical pose classification, in which we formulate the pose estimation as a classification task, and propose a dataset, Yoga-82, for large-scale yoga pose recognition with 82 classes Yoga-82 consists of complex poses where fine annotations may not be possible To resolve this, we provide hierarchical labels for yoga poses based on the body configuration of the pose The dataset contains a three-level hierarchy including body positions, variations in body positions, and the actual pose names We present the classification accuracy of the state-of-the-art convolutional neural network architectures on Yoga-82 We also present several hierarchical variants of DenseNet in order to utilize the hierarchical labels

/pdf/yoga-82-a-new-dataset-for-fine-grained-classification-of-4llq54h7p7.pdf

Yoga-82: A New Dataset for Fine-grained Classification of Human Poses

To have a better understanding and usage of Convolution Neural Networks (CNNs), the visualization and interpretation of CNNs has attracted increasing attention in recent years. In particular, several Class Activation Mapping (CAM) methods have been proposed to discover the connection between CNN's decision and image regions. In spite of the reasonable visualization, lack of clear and sufficient theoretical support is the main limitation of these methods. In this paper, we introduce two axioms -- Conservation and Sensitivity -- to the visualization paradigm of the CAM methods. Meanwhile, a dedicated Axiom-based Grad-CAM (XGrad-CAM) is proposed to satisfy these axioms as much as possible. Experiments demonstrate that XGrad-CAM is an enhanced version of Grad-CAM in terms of conservation and sensitivity. It is able to achieve better visualization performance than Grad-CAM, while also be class-discriminative and easy-to-implement compared with Grad-CAM++ and Ablation-CAM. The code is available at this https URL.

Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs

The retrieval performance of a content-based image retrieval (CBIR) system is mainly influenced by the feature representations and similarity measures. Recently, deep learning develops rapidly and the deep features based on deep learning have been applied widely because it has been shown that the features have very strong generalization. This paper applies the original deep features generated by convolution neural network (CNN) to CBIR, and uses linear support victor machine (SVM) to train a hyerplane which can separate similar image pairs and dissimilar image pairs to a large degree. The input of the SVM in this paper is pair features which are assembled by pair of images: the query image and each test image in the image dataset. The test images then are ranked by the distance between the pair features and the trained hyperplane. Experiments show that our method can significantly improve the performance of CBIR for object image retrieval tasks.

Content-based image retrieval based on CNN and SVM

/pdf/axiom-based-grad-cam-towards-accurate-visualization-and-29i54u4gqt.pdf

Axiom−based Grad−CAM: Towards Accurate Visualization and Explanation of CNNs

Figure-ground segmentation is used to extract the foreground from the background, where the foreground is usually defined as the region containing the most meaningful object of the image. In fact, the algorithms that take advantage of human–computer interaction often attain better performance and they are based on the ‘one-to-one’ model. In this study, the authors present a novel algorithm for figure-ground segmentation based on the GrabCut algorithm, which is a common segmentation algorithm that is user interactive. However, instead of a real user, they attempt to use a pre-trained deep convolutional neural network to interact with GrabCut for completing its job successfully. Weizmann's segmentation evaluation database is used as the test dataset and the results show that their algorithm works well for figure-ground segmentation. While the previous automatic segmentation algorithms are required to rank their segments empirically in order to find the position of the foreground after the segmentation, their algorithm is fully automatic.

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-ipr.2016.0009

Fully automatic figure-ground segmentation algorithm based on deep convolutional neural network and GrabCut

We study the problem of object detection in remote sensing images. As a simple but effective feature extractor, Feature Pyramid Network (FPN) has been widely used in several generic vision tasks. However, it still faces some challenges when used for remote sensing object detection, as the objects in remote sensing images usually exhibit variable shapes, orientations, and sizes. To this end, we propose a dedicated object detector based on the FPN architecture to achieve accurate object detection in remote sensing images. Specifically, considering the variable shapes and orientations of remote sensing objects, we first replace the original lateral connections of FPN with Deformable Convolution Lateral Connection Modules (DCLCMs), each of which includes a 3×3 deformable convolution to generate feature maps with deformable receptive fields. Additionally, we further introduce several Attention-based Multi-Level Feature Fusion Modules (A-MLFFMs) to integrate the multi-level outputs of FPN adaptively, further enabling multi-scale object detection. Extensive experimental results on the DIOR dataset demonstrated the state-of-the-art performance achieved by the proposed method, with the highest mean Average Precision (mAP) of 73.6%.

/pdf/attention-based-multi-level-feature-fusion-for-object-1f5hz9hu.pdf

Ruigang Fu

Papers

Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs

Content-based image retrieval based on CNN and SVM

Axiom−based Grad−CAM: Towards Accurate Visualization and Explanation of CNNs

Fully automatic figure-ground segmentation algorithm based on deep convolutional neural network and GrabCut

Attention-Based Multi-Level Feature Fusion for Object Detection in Remote Sensing Images