This is an Open Access document downloaded from ORCA, Cardiff University's institutional
repository: http://orca.cf.ac.uk/134555/
This is the author’s version of a work that was submitted to / accepted for publication.
Citation for final published version:
Wen, Jingjing, Yao, Houpu, Ji, Ze and Xia, Min 2021. On fault diagnosis for high-g accelerometers
via data-driven models. IEEE Sensors Journal 21 (2) , pp. 1359-1368. 10.1109/JSEN.2020.3019632
file
Publishers page: http://doi.org/10.1109/JSEN.2020.3019632
<http://doi.org/10.1109/JSEN.2020.3019632>
Please note:
Changes made as a result of publishing processes such as copy-editing, formatting and page
numbers may not be reflected in this version. For the definitive version of this publication, please
refer to the published source. You are advised to consult the publisher’s version if you wish to cite
this paper.
This version is being made available in accordance with publisher policies. See
http://orca.cf.ac.uk/policies.html for usage policies. Copyright and moral rights for publications
made available in ORCA are retained by the copyright holders.
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
1
Abstract—Shock test is a pivotal stage for designing and
manufacturing space instruments. As the essential components
in shock test systems to measure shock signals accurately,
high-g accelerometers are usually exposed to hazardous shock
environment and could be subjected to various damages. Owing
to that these damages to the accelerometers could result in
erroneous measurements which would further lead to shock test
failures, accurately diagnosing the fault type of each high-g
accelerometer can be vital to ensure the reliability of the shock
test experiments. Additionally, in practice, an accelerometer in
one malfunction form usually outputs mutable signal
waveforms, so that it is difficult to empirically judge the fault
type of the accelerometer based on the erroneous readings.
Moreover, traditional hardware diagnosis approaches require
disassembling the sensor’s package shell and manually
observing the damage of the elements inner the sensor, which
are less efficient and uneconomical. Aiming at these problems,
several data-driven approaches are incorporated to diagnose
the fault types of high-g accelerometers in this work. Firstly,
several high-g accelerometers with most frequent types of
damage are collected, and a shock signal dataset is gathered by
conducting shock tests on these faulty accelerometers. Then, the
obtained dataset is used to train several base classifiers to
identify the fault types in a supervised fashion. Lastly, a hybrid
ensemble learning model is established by integrating these base
classifiers with both heterogeneous and homogeneous models.
Experimental results show that these data-driven methods can
accurately identify the fault types of high-g accelerometers from
their mutable erroneous readings.
Index Terms—shock test, high-g accelerometer, fault
diagnosis, data-driven methods, ensemble learning.
I. INTRODUCTION
N aircraft/aerospace engineering, the on-board electronic
instruments would undergo multifarious high-g shock
environments [1], such as the release of space equipment
with explosive bolts [2], the impact of orbital debris on
spacecraft structures [3], and the bird striking [4]. With aim
to verify if the space instruments could withstand the severe
shock loads structurally and functionally, various ground
Manuscript submitted April 13, 2020. This work was supported by the
Innovation Foundation for Doctor Dissertation of Northwestern
Polytechnical University (No. CX201902). (Corresponding author: Ze Ji;
Min Xia.)
J. Wen is with the School of Astronautics, Northwestern Polytechnical
University, Xi’an, Shaanxi 710072, China, and also with the School of
Engineering, Cardiff University, Cardiff CF243AA, U.K. (e-mail:
wjj1990@mail.nwpu.edu.cn).
H. Yao is with the JD Finance America Corporation, Mountain View, CA
94043, USA (e-mail: hope-yao@asu.edu).
Z. Ji is with the School of Engineering, Cardiff University, Cardiff
CF243AA, U.K. (e-mail: jiz1@cardiff.ac.uk).
B. Wu is with the School of Astronautics, Northwestern Polytechnical
University, Xi’an, Shaanxi 710072, China (e-mail: wubin@nwpu.edu.cn).
M. Xia is with the Yangtze River Delta Research Institute, Northwestern
Polytechnical University, Taicang, Jiangsu 215400, China (e-mail:
mxia87@gmail.com).
shock test standards have been developed by standardization
bodies, e.g. ISO, JEDEC, IEC, etc., to simulate the real shock
environments, and passing the shock test is an essential
requirement during designing and manufacturing space
instruments [5].
Shock environments are described by acceleration-time
signals generally, which can be measured by accelerometers
[6]. However, high-g accelerometers sometimes can get
damaged during the shock tests due to the severe impact
environment [7]. These damaged accelerometers cannot
measure accurate shock signals, which would lead to
measuring uncertainties of various unpredictable levels, and,
hence, test failures. Additionally, using uncalibrated
accelerometers, but without awareness beforehand, would
result in the test failure as well [8]. There are mainly two
categories of accelerometer faults: the damage of the package
shell and the damage of the inner components [9]. While the
package shell damage can be diagnosed visually, inner
components damage cannot be observed directly.
Accordingly, this work primarily addresses the identification
of the inner components damage. The major damage types of
accelerometer’s inner components include cantilever
fractures, wire bond shearing, solder joint loss, chip cracks,
[10], [11] etc. Correspondingly, inner component damages
will cause the waveform variation of the accelerometer’s
outputs, such as the peak truncation [7], noise pollution [8],
and baseline drift [12]. Therefore, it would be of great value
to be able to automatically diagnose the accelerometer’s fault
type through its readings.
Currently, identifying the fault type of accelerometers
heavily relies on human estimation and prior experience in
signal processing [13], [14]. Such a dependency on human
ingenuity limits the extension for more complicated scenarios
[15]. Additionally, the output signals of a faulty
accelerometer sometimes lack repeatability, and, thus, the
sensors with different fault types are likely to output similar
signal waveforms, which further increases the difficulty of
diagnosing accelerometers’ fault types through their output
readings. Over the past decade, data-driven methods have
achieved great success in the fault diagnosis field based on its
strong feature extraction ability and high performance in
approximating complex functions [16]. Typical fault
diagnosis applications are developed in the fields of rotary
machinery systems [17], battery systems [18], and
engineering structures [19]. All the works not only enrich the
applications of the data-driven methods but also improve the
fault diagnosis level in these fields. However, to the authors’
best knowledge, research in automating fault identification
on shock sensors is still limited, and, hence, the authors are
motivated to investigate in methods for automatic inference
of fault types of accelerometers by directly analyzing their
outputting readings.
On Fault Diagnosis for High-g Accelerometers
via Data-Driven Models
Jingjing Wen, Houpu Yao, Ze Ji, Member, IEEE, Bin Wu, and Min Xia, Member, IEEE
I
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
2
In this work, firstly, a dataset of shock signals from shock
tests are generated by using a high-g shock testing system.
This shock testing system is composed of a drop shock tester
[5] and a dual mass shock amplifier (DMSA) [20], which is
capable of producing high-g shock signals on the order of 1×
10
4
g and conducting shock tests with high efficiency. The
gathered dataset contains 3004 sets of shock signals
measured from one healthy accelerometer and six faulty
accelerometers. For most traditional data driven-based fault
diagnosis methods, all the training datasets and label
information are simulated from laboratory machines, which
could not be representative enough for real-life operating
conditions [21]. In this work, all the faulty types of
accelerometers and all the data are gathered from real-world
experiments. Secondly, five main-stream data-driven models
have been investigated; benchmarked on the collected dataset
to detect different types of sensor faults. The models used in
this paper include multiple-hidden-layer neural networks
(MHLNN), logistic regression (LR), k-nearest neighbor
(k-NN), support vector machine (SVM), and ensemble
learning (EL). Among these models, the Bagging algorithm
is employed to construct the EL model by integrating the
other four models, which are set as the base classifiers.
Lastly, the diagnosis results of the proposed data-driven
methods are visualized and evaluated with several metrics,
including confusion matrix, accuracy, precision, recall,
F
1
-score and computation time. The tested results show that
the proposed data-driven-based diagnosis methods can
effectively identify fault types from mutable shock
waveforms. Especially, the EL model demonstrates superior
identification performance to the other four single
data-driven models.
The major contributions of this work are highlighted
below:
(1) To the authors’ best knowledge, this is the first time to
introduce data-driven methods into fault diagnosis of high-g
accelerometers, hence contributing a new application for
machine learning methods.
(2) A new dataset for industrial shock signals is
established. This dataset contains several typical fault types
of shock sensors and are gathered from real engineering
practice, which will facilitate future research for both fault
diagnosis of high-g accelerometers and signal measurement
in shock tests.
(3) A hybrid EL framework is explored for the fault
diagnosis problem of high-g accelerometers for the first time
by integrating both heterogeneous and homogeneous models,
featuring high accuracy and generalization.
II. PRELIMINARIES OF RELATED DATA-DRIVEN METHODS
The aim of this work is concentrated in learning from
labeled shock signals and predicting the damage type of
accelerometers when a new signal is given. This type of task
falls into the supervised learning category. In this section,
five popular supervised learning methods used in
experiments are reviewed briefly, including MHLNN, LR,
k-NN, SVM, and EL. Without loss of generality, the basic
binary classification problem is used to illustrate the
algorithms for the sake of simplicity. The details of these
methods can be referred in [22].
Neural network (NN) can be viewed as a parametric
function
with
parameters
for each layer and the network
input .
is named as activation function,
which is usually chosen as the ReLU function [23]. The
single hidden layer NN is easy to train and has been widely
used, but its shallow structure limits the ability to further
mine fault information and identify fault types. A typical
deep learning model is MHLNN, which enhances the
capacity of the model by increasing the hidden layers. It has
been proven that, given enough weights, the NN is able to
approximate any complex function. In the training phase, the
optimum weights
can be determined by minimizing the
difference between the NN output and the observations (,
i.e.
, and such optimization
problems can be solved efficiently with the error
back-propagation algorithm.
The LR algorithm is obtained by applying the sigmoid
function and maximum likelihood method to the output of
LR, i.e.,
, where
is the input data; are
the weight parameters;
is the label. In the training phase,
the optimum weights
can be determined by minimizing
, i.e.
, and such optimization
problems can be solved effectively with the convex
optimization theory. The sigmoid function is shown in Fig. 1.
Fig. 1. Illustration of the sigmoid function.
The k-NN algorithm is based on a simple assumption that
similar things are near to each other. A variety of distance
measurements has been proposed to identify the
neighborhood, such as the Euclidian distance, Mahalanobis
distance, and Bhattacharyya distance. In practice, given a
testing data point, the nearest neighbor to it can be found in
the training set and the most frequent label is regarded as the
label to the testing data. An illustration of the Euclidian
distance-based k-NN algorithm with k=3 is shown in Fig. 2.
An advantage of k-NN is that it requires few training
parameters for implementing, which greatly simplifies the
computation of this algorithm.
Fig. 2. Illustration of k-NN algorithm with k=3.
The SVM method is an elegant algorithm and has achieved
great success since the 1990s. As shown in Fig. 3, the idea of
SVM is to find a hyper-plane in the high dimensional feature
space that distinctly classifies the training data. Data falls into
different sides of the hyper-plane is regarded as different
classes. A nice convex optimization problem can be
formulated under such a problem setup to find the support
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
3
vectors, allowing fast solution and easy error estimation.
Different kernel functions have been introduced to extend
this algorithm, making SVM handle nonlinear separable
cases well [24].
Fig. 3. Illustration of the SVM method (different colors represent different
classes; the filled squares and circles correspond to the support vectors of the
dataset).
As shown in Fig. 4, EL is a machine learning approach
which can obtain better classification accuracy and
generalization by combining multiple base classifiers.
Generally, the base classifiers are simple with little
computation, and the diversities among these base classifiers
are the primary requirement instead of the classification
accuracy [25]. Theoretically, the classification performance
of the integrated classifier trained with EL is better than that
of each stand-alone base classifier [25]. Base classifiers can
be of the same type or different types, termed as
homogeneous ensemble models and heterogeneous ensemble
models respectively [26]. According to the strength of the
dependencies between the base classifiers, EL method can be
roughly divided into two categories: serial and parallel
generation methods. The former adapts to the condition that
there are strong dependencies between the base classifiers,
and the representative strategy is the Boosting algorithm.
Oppositely, the latter applies to the situation that there are
weak dependencies between base classifiers, and the typical
approach is the Bagging algorithm. In the last few decades,
EL methods have been successfully employed for the fault
diagnosis in various fields, including rotary machineries [25],
gas turbine engines [26], photovoltaic systems [27], etc.
Fig. 4. Schematic diagram of EL method.
III. EXPERIMENTAL SYSTEM AND DATA COLLECTION
In many previous studies for fault diagnosis, fault types are
simulated using laboratory machines in some predefined
conditions to mimic the fault behaviors [17], [28]. Lei and Lu
et al. point out that the fault diagnosis knowledge from
laboratory machines is different from real-case machines [21],
[29]. In this work, all the tested accelerometers are collected
from actual practice, and the experimental system is a
standard shock test setup which can produce real shock test
environments.
Fig. 5 displays the high-g shock test system used in this
work to generate the dataset and verify the performance of
the proposed data-driven methods. This system combines a
drop shock tester [5] and a DMSA [20]. The working
principle of this platform is as follows: Firstly, lift the drop
table and the DMSA up to a drop height and then release
them in free fall together; Secondly, a strong collision will
occur with the drop table falling on the rubber programmer
producing a primary impact, and bounce upward due to the
programmer’s elasticity; Lastly, the DMSA table will
continue moving downward and collide with the
upward-moving DMSA base in a secondary impact,
producing a high-g shock. In this system, different drop
heights will generate different shock levels, and a dataset
containing massive shock signals with different shock levels
can be obtained by repeating these procedures. This system
can be used as a standard device for high-g shock tests
directly, and, thus, the collected dataset is highly consistent
with the practical working conditions of real shock test.
In this work, one functional accelerometer and six faulty
accelerometers, which were all collected from practical shock
tests, are used to measure shock signals simultaneously. As
shown in Fig. 5, these accelerometers are connected to the
charge amplifier with signal lines. After amplification, the
shock signals are transmitted into the data acquisition and
processing system through a shielded signal cable. The
sampling frequency is 200 kHz, which guarantees the high
fidelity of the measurements. Then the processed shock
signals can be visualized on the monitor directly.
Fig. 5. Experimental system. More details can be found in [7].
Besides, in order to eliminate the incommensurability
between different types of sensors, the used accelerometers
are all the piezoelectric type. As shown in Fig. 6, the
piezoelectric accelerometer is generally composed of the
shell, a mass block, a piezoelectric element, and so on. The
working principle is introduced briefly here. The base of the
accelerometer is fixed on an object rigidly with the mounting
bolt, and move at the same acceleration with the object.
When conducting measurement, the piezoelectric element is
subjected to the inertial force of the mass block opposite to
the acceleration direction, and an alternating charge is
generated on both surfaces of the piezoelectric element. After
amplification, the value of the charge can be measured by the
measuring instrument, and the acceleration of the object is
obtained [30].
In this work, a total of 751 times shock tests with different
drop heights were conducted and 3004 sets of shock signals
were collected. In order to acquire the shock information as
accurate as possible, no pre-processing was carried out on
these signals. The typical readings from all the
accelerometers under different fault types are displayed in
Fig. 7. These typical signal readings were picked out
manually and empirically. It can be seen from Fig. 7 that,
except accelerometers 1 and 3, every other faulty
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
4
accelerometer with one fault type demonstrates mutable
faulty signal waveforms, and different faulty sensors could
output similar faulty waveforms too. The detailed description
for the faulty readings of each accelerometer is summarized
in Table I. Therefore, it is considered highly challenging to be
able to accurately identify the fault type of the accelerometer
from its erroneous readings empirically. This phenomenon
that sensors with different fault types output similar readings
could be caused by the low repeatability of the faulty sensors.
Besides, it is worth noting that these erroneous shock
waveforms are measured from the accelerometers used in this
work, but in practice, even with the same fault type, different
signal waveforms can be produced under different
experimental environments and different sensor types.
Fig. 6. Schematic diagram of piezoelectric accelerometer’s structure.
IV. TRAINING AND TESTING OF THE DATA-DRIVEN
METHODS
A. Training and Testing of the Base Classifiers
In the gathered 3004 sets of shock signals, 80% of the
samples for each accelerometer were randomly extracted as
the training dataset, and the remaining 20% were maintained
as the test dataset. Four different models, including MHLNN,
k-NN, LR, and SVM, are trained on the dataset, and the fault
types of these accelerometers are monitored by classification.
The MHLNN used in this paper has three hidden layers,
with 512, 32 and 8 nodes respectively. The MHLNN is
trained for 200 epochs, and the training process for the
proposed MHLNN is monitored in Fig. 8. The Adam
optimizer with a learning rate of
and batch size of
32 was used. It can be seen that both training and testing
accuracy are converging well.
The number of neighbors used in the k-NN algorithm is
evaluated with values ranging from 2 to 9. The testing
accuracy for the k-NN models with different numbers of
neighbor is monitored to indicate the performance of the
corresponding k values. As shown in Fig. 9, the accuracy first
increases and then decreases with the increasing neighbors.
This is due to that the model is over-fitting with smaller k and
tends to be under-fitting with larger k. Besides, it can be seen
that the k-NN model gets the optimal accuracy when k equals
to 3.
The gradient descent method is applied in the LR
algorithm to calculate the optimum weights. The radial basis
function is used as the kernel in the SVM algorithm. The
kernel parameters and penalty factors are optimized by the
least square method.
B. Diversity Measures of the Base Classifiers
The primary goal of EL is to improve the performance of a
model by aggregating multiple weak classifiers. In EL, the
diversity among base classifiers plays an essential role for
constructing effective ensemble systems. Kuncheva et al.
summarize 10 typical measures of diversity [31], among
which the Q statistics is selected as the diversity measure in
this work. The calculation method of Q statistics is expressed
as follows: Assume that the number of the base classifiers is
L
c
; C
i
and C
j
(i, j=1, 2, …… , L
c
, i≠j) are two different base
classifiers; N
11
(N
00
) is the number of samples that C
i
and C
j
both classify correctly (incorrectly); N
10
(N
01
) is the number of
samples that meet the following requirements: C
i
(C
j
)
classifies them correctly and C
j
(C
i
) misclassifies them. The
relationship between a pair of classifiers is shown in Table Ċ,
and the calculation of Q statistics can be expressed as:
. (1)
It can be seen from (1) that, if the two classifiers demonstrate
the same classification results, i.e. N
10
=N
01
=0, then Q=1, and
the diversity between these two classifiers is the lowest.
Oppositely, if the two classifiers have different classification
results on each sample, i.e. N
11
=N
00
=0, then Q= −1, in which
the diversity is the highest. Furthermore, the diversity of the
whole ensemble system can be calculated with
. (2)
C. Training and Testing of the Heterogeneous EL Model
A heterogeneous EL model is established by importing the
Bagging strategy into MHLNN, k-NN (k=3), LR, and SVM.
The Bagging algorithm is one of the representatives of the
parallel EL method [32]. As illustrated in Fig. 10, the
procedure of constructing an EL model with Bagging is that:
Firstly, the Bootstrap method is utilized to randomly
resample the samples in the dataset with replacement to attain
multiple sub-datasets with same size. The probability of the
samples not being selected in a sub-dataset can be expressed
as
, (3)
where is the number of samples in the dataset. When
tends to infinity, the limit value of is around 36.8%, and,
accordingly, the probability of the samples being selected is
63.2% approximately. The selected samples, termed as
in-bag (IB) data, are used to train the EL model, while the
unselected samples, termed as out-of-bag (OOB) data, can be
used as the validation data to estimate the accuracy and
generalization of the EL model. Secondly, in this work, four
sub-datasets are generated and used to train the base
classifiers respectively. Lastly, the final classification result
of the EL model can be determined by aggregating the
outputs of all the base classifiers with the majority voting
strategy.
In the majority voting strategy, the final classification is
decided based on the agreement of more than half of the base
classifiers. This EL model can be parallelized to accelerate
the computation. This EL model is an open system, which
can be further improved by integrating any other base
classifiers, such as decision tree, deep belief network, and
convolution residual network. Additionally, with aim to
compare the classification performance between the EL
model and the other data-driven models conveniently, the
OOB data are also designed to account for 20% of all the
samples in the dataset.