The intelligent diagnosis of wheel flat based on vibration image classification is a promising research subject for performance maintenance of railway vehicles. However, the image representation method of vibration signal and classification network construction under small samples have become two obstacles to intelligent diagnosis of wheel flat. This paper presents a novel frequency-domain Gramian angular field (FDGAF) algorithm to encode the vibration signal of wheel flat to featured images. Furthermore, a modified transfer learning network is introduced to classify these featured images under small samples without any involvement of prior knowledge. The proposed FDGAF can calculate the Gramian angular matrix of axle box acceleration signal in frequency domain and assign frequency position dependence to the featured images to preserve original characteristic information. Then, these featured images can be intelligent classified by a transfer learning network under the condition of 30 sample without require of prior knowledge. To verify the efficiency of this proposed method, 12 cases of artificial wheel flats are processed on a scaled railway test rig, and their axle box acceleration signals are collected to obtain visual diagnosis results. The verfication proves that FDGAF is able to obtain accurate diagnostic results with high separability, for separability indexes of FDGAF reaches 10.8, 8.7, 14.9, and 5.8. We anticipate that this method will find use in the performance maintenance of railway vehicles and the improvement of industrial condition monitoring.

/pdf/intelligent-diagnosis-for-railway-wheel-flat-using-frequency-3e07730iwd.pdf

Intelligent Diagnosis for Railway Wheel Flat Using Frequency-Domain Gramian Angular Field and Transfer Learning Network

In this paper, we present a novel approximate computing scheme suitable for realizing the energy-efficient multiply-accumulate (MAC) processing. In contrast to the prior works that suffer from the error accumulation limiting the approximate range, we utilize different approximate multipliers in an interleaved way to compensate errors in the opposite direction during accumulate operations. For the balanced error accumulation, we first design the approximate 4–2 compressors generating errors in the opposite direction while minimizing the computational costs. Based on the probabilistic analysis, positive and negative multipliers are then carefully developed to provide a similar error distance. Simulation results on various practical applications reveal that the proposed MAC processing offers the energy-efficient computing scenario by extending the range of approximate parts. Even compared to the state-of-the-art solutions, for example, the proposed interleaving scheme relaxes the core-level energy consumption of the recent CNN accelerator by more than 35% without degrading the recognition accuracy.

Design and Analysis of Approximate Compressors for Balanced Error Accumulation in MAC Operator

Weld defect recognition plays an important role in the manufacturing process of large-scale equipment. Traditional methods generally include several serial steps, such as image preprocessing, region segmentation, feature extraction, and type recognition. The results of each step have significant impact on the accuracy of the final defect identification. The convolutional neural network (CNN) has strong pattern recognition ability, which can overcome the above problem. However, there are two problems: one is that the pooling strategy has poor dynamic adaptability and the other is the insufficient feature selection ability. To overcome these problems, we propose a CNN-based weld defect recognition method, which includes an improved pooling strategy and an enhanced feature selection method. According to the characteristics of the weld defect image, an improved pooling strategy that considers the distribution of the pooling region and feature map is introduced. Additionally, in order to enhance the feature selection ability of the CNN, an enhanced feature selection method integrating the ReliefF algorithm with the CNN is proposed. A case study is presented for demonstrating the proposed techniques. The results show that the proposed method has higher accuracy than the traditional CNN method, and establish that the proposed CNN-based method is successfully applied for weld defect recognition.

Convolution neural network model with improved pooling strategy and feature selection for weld defect recognition

This paper presents a novel stacking and multi-level indexing scheme for convolutional neural networks (CNNs) used in energy-limited edge-level systems. Basically, the proposed scheme offers multiple accuracy modes by adopting a structured weight pruning method that enables a CNN to be trained once with multiple pruning ratios and thereby allows for adaptive energy-accuracy trade-offs. The memory overhead required to store several different networks is kept to a minimum by adopting a novel method for including smaller lower-accuracy networks as subnetworks of larger higher-accuracy networks and by using a unique multi-level indexing scheme that can effectively store compressed weight data for the proposed stacked-CNN architecture. Experimental results show that the proposed method successfully reduces the memory footprint by up to 33% when compared to a baseline CNN architecture. An FPGA-based multi-mode CNN accelerator that implements the proposed scheme has been designed. Energy usage analysis with a case study shows that the inference energy required for on-device CNN processing can be reduced by up to 1.94 times over the baseline design.

Memory-Reduced Network Stacking for Edge-Level CNN Architecture With Structured Weight Pruning

This paper presents an approximate computing method of long short-term memory (LSTM) operations for energy-efficient end-to-end speech recognition. We newly introduce the concept of similarity score, which can measure how much the inputs of two adjacent LSTM cells are similar to each other. Then, we disable the highly-similar LSTM operations and directly transfer the prior results for reducing the computational costs of speech recognition. The pseudo-LSTM operation is additionally defined for providing the approximate computation with reduced processing resolution, which can further relax the processing overheads without degrading the accuracy. In order to verify the proposed idea, in addition, we design an approximate LSTM accelerator in 65 nm CMOS process. The proposed accelerator newly utilizes a number of approximate processing elements (PEs) to support the proposed skipped-LSTM and pseudo-LSTM operations without degrading the energy efficiency. Moreover, sparsity-aware scheduling is introduced by introducing the small-sized on-chip SRAM buffer. As a result, the proposed work provides an energy-efficient but still accurate speech recognition system, which consumes 2.19 times less energy than the baseline architecture.

https://www.mdpi.com/2079-9292/9/12/2004/pdf

Approximate LSTM Computing for Energy-Efficient Speech Recognition

Neural networks trained using images with a certain type of distortion should be better at classifying test images with the same type of distortion than generally-trained neural networks, given other factors being equal. Based on this observation, an ensemble of convolutional neural networks (CNNs) trained with different types and degrees of distortions is used. However, instead of simply classifying test images of unknown distortion types with the entire ensemble of CNNs, an extra tiny CNN is specifically trained to distinguish between the different types and degrees of distortions. Then, only the dedicated CNN for that specific type and degree of distortion, as determined by the tiny CNN, is activated and used to classify a possibly distorted test image. This proposed architecture, referred to as a selective deep convolutional neural network (DCNN), is implemented and found to result in high accuracy with low hardware costs. Detailed simulations with realistic image distortion scenarios using three popular datasets show that memory, MAC operations, and energy savings of up to 93.68%, 93.61%, and 91.92%, respectively, can be achieved with almost no reduction in image classification accuracy. The proposed selective DCNN scores up to $2.18\times $ higher than the state-of-the-art DCNN model when evaluated using NetScore, a comprehensive metric that considers both CNN performance and hardware cost. In addition, it is shown that even higher hardware cost reduction can be achieved when selective DCNN is combined with previously proposed model compression techniques. Finally, experiments conducted with extended types and degrees of image distortion show that selective DCNN is highly scalable.

/pdf/selective-deep-convolutional-neural-network-for-low-cost-gxtjxv5oib.pdf

Selective Deep Convolutional Neural Network for Low Cost Distorted Image Classification

In this paper, we present a novel channel scaling scheme for convolutional neural networks (CNNs), which can improve the recognition accuracy for the practical distorted images without increasing the network complexity. During the training phase, the proposed work first prepares multiple filters under the same CNN architecture by taking account of different noise models and strengths. We then newly introduce an FFT-based noise classifier, which determines the noise property in the received input image by calculating the partial sum of the frequency-domain values. Based on the detected noise class, we dynamically change the filters of each CNN layer to provide the dedicated recognition. Furthermore, we propose a channel scaling technique to reduce the number of active filter parameters if the input data is relatively clean. Experimental results show that the proposed dynamic channel scaling reduces the computational complexity as well as the energy consumption, still providing the acceptable accuracy for intelligent edge devices.

Low-Complexity Dynamic Channel Scaling of Noise-Resilient CNN for Intelligent Edge Devices

Power consumed by a computer memory system can be significantly reduced if a certain level of error is permitted in the data stored in memory. Such an approximate memory approach is viable for use in applications developed using deep neural networks (DNNs) because such applications are typically error-resilient. In this paper, the use of hierarchical approximate memory for DNNs is studied and modeled. Although previous research has focused on approximate memory for specific memory technologies, this work proposes to consider approximate memory for the entire memory hierarchy of a computer system by considering the error budget for a given target application. This paper proposes a system model based on the error budget (amount by which the memory error rate can be permitted to rise to) for a target application and the power usage characteristics of the constituent memory technologies of a memory hierarchy. Using DNN case studies involving SRAM, DRAM, and NAND, this paper shows that the overall memory power consumption can be reduced by up to 43.38% by using the proposed model to optimally divide up the available error budget.

Hierarchical Approximate Memory for Deep Neural Network Applications

One of the most important objectives of post-marketing monitoring of dietary supplements is the early detection of unknown and unexpected adverse events (AEs). Several causality algorithms, such as the Naranjo scale, the RUCAM scale, and the M & V scale are available for the estimation of the likelihood of causation between a product and an AE. Based on the existing algorithms, the Korea Food & Drug Administration has developed a new algorithm tool to reflect the characteristics of dietary supplements in the causality analysis. However, additional work will be required to confirm if the newly developed algorithm tool has reasonable sensitivity and not to generate an unacceptable number of false positives signals.

Jeonghun Kim

Papers

Selective Deep Convolutional Neural Network for Low Cost Distorted Image Classification

Low-Complexity Dynamic Channel Scaling of Noise-Resilient CNN for Intelligent Edge Devices

Hierarchical Approximate Memory for Deep Neural Network Applications

Algorithms for Causality Evaluation of Adverse Events from Health/Functional Foods