Recent advances in convolutional neural networks

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation. Besides, we also introduce various applications of convolutional neural networks in computer vision, speech and natural language processing.

Recent Advances in Convolutional Neural Networks

The performance of most conventional classification systems relies on appropriate data representation and much of the efforts are dedicated to feature engineering, a difficult and time-consuming process that uses prior expert domain knowledge of the data to create useful features. On the other hand, deep learning can extract and organize the discriminative information from the data, not requiring the design of feature extractors by a domain expert. Convolutional Neural Networks (CNNs) are a particular type of deep, feedforward network that have gained attention from research community and industry, achieving empirical successes in tasks such as speech recognition, signal processing, object recognition, natural language processing and transfer learning. In this paper, we conduct some preliminary experiments using the deep learning approach to classify breast cancer histopathological images from BreaKHis, a publicly dataset available at http://web.inf.ufpr.br/vri/breast-cancer-database. We propose a method based on the extraction of image patches for training the CNN and the combination of these patches for final classification. This method aims to allow using the high-resolution histopathological images from BreaKHis as input to existing CNN, avoiding adaptations of the model that can lead to a more complex and computationally costly architecture. The CNN performance is better when compared to previously reported results obtained by other machine learning models trained with hand-crafted textural descriptors. Finally, we also investigate the combination of different CNNs using simple fusion rules, achieving some improvement in recognition rates.

Breast cancer histopathological image classification using Convolutional Neural Networks

Selection of text feature item is a basic and important matter for text mining and information retrieval. Traditional methods of feature extraction require handcrafted features. To hand-design, an effective feature is a lengthy process, but aiming at new applications, deep learning enables to acquire new effective feature representation from training data. As a new feature extraction method, deep learning has made achievements in text mining. The major difference between deep learning and conventional methods is that deep learning automatically learns features from big data, instead of adopting handcrafted features, which mainly depends on priori knowledge of designers and is highly impossible to take the advantage of big data. Deep learning can automatically learn feature representation from big data, including millions of parameters. This thesis outlines the common methods used in text feature extraction first, and then expands frequently used deep learning methods in text feature extraction and its applications, and forecasts the application of deep learning in feature extraction.

/pdf/text-feature-extraction-based-on-deep-learning-a-review-tvbjw7vvga.pdf

Text feature extraction based on deep learning: a review

In many water distribution systems, a significant amount of water is lost because of leakage during transit from the water treatment plant to consumers. As a result, water leakage detection and localization have been a consistent focus of research. Typically, diagnosis or detection systems based on sensor signals incur significant computational and time costs, whereas the system performance depends on the features selected as input to the classifier. In this paper, to solve this problem, we propose a novel, fast, and accurate water leakage detection system with an adaptive design that fuses a one-dimensional convolutional neural network and a support vector machine. We also propose a graph-based localization algorithm to determine the leakage location. An actual water pipeline network is represented by a graph network and it is assumed that leakage events occur at virtual points on the graph. The leakage location at which costs are minimized is estimated by comparing the actual measured signals with the virtually generated signals. The performance was validated on a wireless sensor network based test bed, deployed on an actual WDS. Our proposed methods achieved 99.3% leakage detection accuracy and a localization error of less than 3 m.

Xiao-Xiao Niu

Papers

A novel hybrid CNN-SVM classifier for recognizing handwritten digits