Application of deep learning algorithms in geotechnical engineering: a short critical review
TL;DR: This study presented the state of practice of DL in geotechnical engineering, and depicted the statistical trend of the published papers, as well as describing four major algorithms, including feedforward neural, recurrent neural network, convolutional neural network and generative adversarial network.
Abstract: With the advent of big data era, deep learning (DL) has become an essential research subject in the field of artificial intelligence (AI). DL algorithms are characterized with powerful feature learning and expression capabilities compared with the traditional machine learning (ML) methods, which attracts worldwide researchers from different fields to its increasingly wide applications. Furthermore, in the field of geochnical engineering, DL has been widely adopted in various research topics, a comprehensive review summarizing its application is desirable. Consequently, this study presented the state of practice of DL in geotechnical engineering, and depicted the statistical trend of the published papers. Four major algorithms, including feedforward neural (FNN), recurrent neural network (RNN), convolutional neural network (CNN) and generative adversarial network (GAN) along with their geotechnical applications were elaborated. In addition, a thorough summary containing pubilished literatures, the corresponding reference cases, the adopted DL algorithms as well as the related geotechnical topics was compiled. Furthermore, the challenges and perspectives of future development of DL in geotechnical engineering were presented and discussed.
Citations
More filters
01 Jan 1983
TL;DR: The neocognitron recognizes stimulus patterns correctly without being affected by shifts in position or even by considerable distortions in shape of the stimulus patterns.
Abstract: Suggested by the structure of the visual nervous system, a new algorithm is proposed for pattern recognition. This algorithm can be realized with a multilayered network consisting of neuron-like cells. The network, “neocognitron”, is self-organized by unsupervised learning, and acquires the ability to recognize stimulus patterns according to the differences in their shapes: Any patterns which we human beings judge to be alike are also judged to be of the same category by the neocognitron. The neocognitron recognizes stimulus patterns correctly without being affected by shifts in position or even by considerable distortions in shape of the stimulus patterns.
649 citations
TL;DR: Three Machine Learning-based methods of Modified Decision Tree (MDT), LightGBM, and XGBoost regressions are proposed to predict construction equipment's residual value to help advancing automation as a coherent field of research within the construction industry.
88 citations
TL;DR: Wang et al. as discussed by the authors developed an ensemble learning-based method to predict the slope stability by introducing the random forest (RF) and extreme gradient boosting (XGBoost), which is applied to the stability prediction of 786 landslide cases in Yunyang County, Chongqing, China.
Abstract: Slope stability prediction plays a significant role in landslide disaster prevention and mitigation. This study develops an ensemble learning-based method to predict the slope stability by introducing the random forest (RF) and extreme gradient boosting (XGBoost). As an illustration, the proposed approach is applied to the stability prediction of 786 landslide cases in Yunyang County, Chongqing, China. For comparison, the predictive performance of RF, XGBoost, support vector machine (SVM), and logistic regression (LR) is systematically investigated based on the well-established confusion matrix, which contains the known indices of recall rate, precision, and accuracy. Furthermore, the feature importance of the 12 influencing variables is also explored. Results show that the accuracy of the XGBoost and RF for both the training and testing data is superior to that of SVM and LR, revealing the superiority of the ensemble learning models (i.e. XGBoost and RF) in the slope stability prediction of Yunyang County. Among the 12 influencing factors, the profile shape is the most important one. The proposed ensemble learning-based method offers a promising way to rationally capture the slope status. It can be extended to the prediction of slope stability of other landslide-prone areas of interest.
77 citations
Posted Content•
TL;DR: A taxonomy is proposed and classify the methods into six sub-categories according to the ways of utilizing inter-frame information for video super-resolution, to alleviate understandability and transferability of existing and future techniques into practice.
Abstract: In recent years, deep learning has made great progress in many fields such as image recognition, natural language processing, speech recognition and video super-resolution. In this survey, we comprehensively investigate 33 state-of-the-art video super-resolution (VSR) methods based on deep learning. It is well known that the leverage of information within video frames is important for video super-resolution. Thus we propose a taxonomy and classify the methods into six sub-categories according to the ways of utilizing inter-frame information. Moreover, the architectures and implementation details of all the methods are depicted in detail. Finally, we summarize and compare the performance of the representative VSR method on some benchmark datasets. We also discuss some challenges, which need to be further addressed by researchers in the community of VSR. To the best of our knowledge, this work is the first systematic review on VSR tasks, and it is expected to make a contribution to the development of recent studies in this area and potentially deepen our understanding to the VSR techniques based on deep learning.
72 citations
Cites background from "Application of deep learning algori..."
...In (Zhang et al., 2012), HR 4D computerized tomography (CT) images are super-resolved with several frames for each slice at different respiratory phases....
[...]
TL;DR: A new technique of intelligence system based on the adaptive neuro-fuzzy inference system (ANFIS)-group method of data handling (GMDH) optimized by the imperialism competitive algorithm (ICA) for forecasting pile bearing capacity is presented.
Abstract: The pile bearing capacity is considered as the most essential factor in designing deep foundations. Direct determination of this parameter in site is costly and difficult. Hence, this study presents a new technique of intelligence system based on the adaptive neuro-fuzzy inference system (ANFIS)-group method of data handling (GMDH) optimized by the imperialism competitive algorithm (ICA), ANFIS-GMDH-ICA for forecasting pile bearing capacity. In this advanced structure, the ICA role is to optimize the membership functions obtained by ANFIS-GMDH technique for receiving a higher accuracy level and lower error. To develop this model, the results of 257 high strain dynamic load tests (performed by authors) were considered and used in the analysis. For comparison purposes, ANFIS and GMDH models were selected and built for pile bearing capacity estimation. In terms of model accuracy, the obtained results showed that the newly developed model (i.e., ANFIS-GMDH-ICA) receives more accurate predicted values of pile bearing capacity compared to those obtained by ANFIS and GMDH predictive models. The proposed ANFIS-GMDH-ICA can be utilized as an advanced, applicable and powerful technique in issues related to foundation engineering and its design.
50 citations
References
More filters
27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
123,388 citations
Proceedings Article•
03 Dec 2012TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Abstract: We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
73,978 citations
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Abstract: Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
72,897 citations
01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Abstract: Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
42,067 citations
08 Dec 2014
TL;DR: A new framework for estimating generative models via an adversarial process, in which two models are simultaneously train: a generative model G that captures the data distribution and a discriminative model D that estimates the probability that a sample came from the training data rather than G.
Abstract: We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to ½ everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.
38,211 citations