Outdoor insulators may experience stress due to severe environmental conditions, such as pollution and contamination. Through the identification of partial discharges by ultrasonic noise, it is possible to assess the possibility of a power grid failure occurring. In this paper, ensemble models are used to analyze an ultrasonic signal from an ultrasonic microphone Pettersson M500. As the insulators are susceptible to developing irreversible failures, it will be evaluated whether the ultrasonic signal will remain over time, so that it is possible to assess whether the discharges being captured can result in a failure in contaminated polymeric insulators, evaluated in a high voltage laboratory under controlled conditions. The ensemble models were used in this paper because they typically require less computational effort than techniques based on deep learning and have acceptable performance for the problem at hand. The bagging, boosting, random subspace, bagging plus random subspace, and stacked generalization ensemble models are evaluated, and the best result of each model is used to compare the differences between the models. The bagging ensemble learning model proved to be faster and have lower error than other ensemble models, long short-term memory (LSTM), and nonlinear autoregressive (NAR). 

https://ieeexplore.ieee.org/ielx7/6287639/9668973/09739678.pdf

Analysis of the Ultrasonic Signal in Polymeric Contaminated Insulators Through Ensemble Learning Methods

Outdoor insulators may experience stress due to severe environmental conditions, such as pollution and contamination. Through the identification of partial discharges by ultrasonic noise, it is possible to assess the possibility of a power grid failure occurring. In this paper, ensemble models are used to analyze an ultrasonic signal from an ultrasonic microphone Pettersson M500. As the insulators are susceptible to developing irreversible failures, it will be evaluated whether the ultrasonic signal will remain over time, so that it is possible to assess whether the discharges being captured can result in a failure in contaminated polymeric insulators, evaluated in a high voltage laboratory under controlled conditions. The ensemble models were used in this paper because they typically require less computational effort than techniques based on deep learning and have acceptable performance for the problem at hand. The bagging, boosting, random subspace, bagging plus random subspace, and stacked generalization ensemble models are evaluated, and the best result of each model is used to compare the differences between the models. The bagging ensemble learning model proved to be faster and have lower error than other ensemble models, long short-term memory (LSTM), and nonlinear autoregressive (NAR).

Analysis of the ultrasonic signal in polymeric contaminated insulators through ensemble learning methods

Due to their increasing spread, confidence in neural network predictions became more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over or under confidence. Many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identified and a variety of approaches to measure and quantify uncertainty in neural networks have been proposed. This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. A comprehensive introduction to the most crucial sources of uncertainty is given and their separation into reducible model uncertainty and not reducible data uncertainty is presented. The modeling of these uncertainties based on deterministic neural networks, Bayesian neural networks, ensemble of neural networks, and test-time data augmentation approaches is introduced and different branches of these fields as well as the latest developments are discussed. For a practical application, we discuss different measures of uncertainty, approaches for the calibration of neural networks and give an overview of existing baselines and implementations. Different examples from the wide spectrum of challenges in different fields give an idea of the needs and challenges regarding uncertainties in practical applications. Additionally, the practical limitations of current methods for mission- and safety-critical real world applications are discussed and an outlook on the next steps towards a broader usage of such methods is given.

A Survey of Uncertainty in Deep Neural Networks.

The learning rate is an important hyperparameter for training deep neural networks. The traditional learning rate method has the problems of instability of accuracy. Aiming at these problems, we proposed a new learning rate method with different cyclical changes in each training cycle instead of a fixed value. It achieves higher accuracy in less iterations and faster convergence. Through the experiment on CIFAR-10 and CIFAR-100 datasets based on VGG network and RESNET network, the final results show that the proposed method has better results on stability and accuracy than cyclical learning rate method.

A Cyclical Learning Rate Method in Deep Learning Training

The ensemble learning method is a necessary process that provides robustness and is more accurate than the single model. The snapshot ensemble convolutional neural network (CNN) has been successful and widely used in many domains, such as image classification, fault diagnosis, and plant image classification. The advantage of the snapshot ensemble CNN is that it combines the cyclic learning rate schedule in the algorithm to snap the best model in each cycle. In this research, we proposed the dropCyclic learning rate schedule, which is a step decay to decrease the learning rate value in every learning epoch. The dropCyclic can reduce the learning rate and find the new local minimum in the subsequent cycle. We evaluated the snapshot ensemble CNN method based on three learning rate schedules: cyclic cosine annealing, max-min cyclic cosine learning rate scheduler, and dropCyclic then using three backbone CNN architectures: MobileNetV2, VGG16, and VGG19. The snapshot ensemble CNN methods were tested on three aerial image datasets: UCM, AID, and EcoCropsAID. The proposed dropCyclic learning rate schedule outperformed the other learning rate schedules on the UCM dataset and obtained high accuracy on the AID and EcoCropsAID datasets. We also compared the proposed dropCyclic learning rate schedule with other existing methods. The results show that the dropCyclic method achieved higher classification accuracy compared with other existing methods. 

/pdf/dropcyclic-snapshot-ensemble-convolutional-neural-network-2hz4fdbc.pdf

dropCyclic: Snapshot Ensemble Convolutional Neural Network Based on a New Learning Rate Schedule for Land Use Classification

Ensembling deep learning models is a shortcut to promote its implementation in new scenarios, which can avoid tuning neural networks, losses and training algorithms from scratch. However, it is difficult to collect sufficient accurate and diverse models through once training. This paper proposes Auto-Ensemble (AE) to collect checkpoints of deep learning model and ensemble them automatically by adaptive learning rate scheduling algorithm. The advantage of this method is to make the model converge to various local optima by scheduling the learning rate in once training. When the number of local optimal solutions tends to be saturated, all the collected checkpoints are used for ensemble. Our method is universal, it can be applied to various scenarios. Experiment results on multiple datasets and neural networks demonstrate it is effective and competitive, especially on few-shot learning. Besides, we proposed a method to measure the distance among models. Then we can ensure the accuracy and diversity of collected models.

Fei Wang

Papers

Auto-Ensemble: An Adaptive Learning Rate Scheduling Based Deep Learning Model Ensembling