Liaqat, S., Dashtipour, K., Arshad, K., Assaleh, K. and Ramzan, N. (2021) A hybrid
posture detection framework: Integrating machine learning and deep neural
networks. IEEE Sensors Journal, (doi: 10.1109/JSEN.2021.3055898)
There may be differences between this version and the published version. You are
advised to consult the publisher’s version if you wish to cite from it.
http://eprints.gla.ac.uk/233291/
Deposited on 04 February 2021
Enlighten – Research publications by members of the University of Glasgow
http://eprints.gla.ac.uk
1530-437X (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2021.3055898, IEEE Sensors
Journal
IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017 1
A hybrid posture detection framework:
Integrating machine learning and deep neural
networks
Sidrah Liaqat, Kia Dashtipour, Kamran Arshad, Khaled Assaleh, Naeem Ramzan
Abstract— The posture detection received lots of attention in the
fields of human sensing and artificial intelligence. Posture detection
can be used for the monitoring health status of elderly remotely
by identifying their postures such as standing, sitting and walk-
ing. Most of the current studies used traditional machine learning
classifiers to identify the posture. However, these methods do not
perform well to detect the postures accurately. Therefore, in this
study, we proposed a novel hybrid approach based on machine
learning classifiers (i. e., support vector machine (SVM), logistic
regression (KNN), decision tree, Naive Bayes, random forest, Lin-
ear discrete analysis and Quadratic discrete analysis) and deep
learning classifiers (i. e., 1D-convolutional neural network (1D-CNN),
2D-convolutional neural network (2D-CNN), LSTM and bidirectional
LSTM) to identify posture detection. The proposed hybrid approach
uses prediction of machine learning (ML) and deep learning (DL)
to improve the performance of ML and DL algorithms. The experi-
mental results on widely benchmark dataset are shown and results
achieved an accuracy of more than 98%.
Index Terms— Posture detection, Hybrid Approach,
Deep Learning, Machine Learning
I. INTRODUCTION
The posture detection is used in different applications such
as healthcare, surveillance, virtual environment, indoor and
outdoor monitoring, the reality for animation and entertain-
ment [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. In addition, the posture
detection can be used in framework of home-human interface.
With the increased number of elderly population and limited
healthcare resources, it is important to propose a technology
which can support the remote monitoring of elderly and
vulnerable people to live more independently [11, 12, 13].
Maintain the good posture is significant to lead the healthy
life. The posture is about how the people hold their body and
position the limbs. Within the advancement of the technology,
the human has chosen the sedentary lifestyle which leading
to less physical activity and movement [14, 15, 16, 17, 18,
19]. The long time sitting during the work or study leads
to decrease in muscle strength. The sedentary lifestyle have
Sidrah Liaqat and Naeem Ramzan are with School of Engineering
and Computing, University of the West of Scotland, Paisely PA1 2BE,
UK.(E-mail: sidra193@gmail.com, Naeem.Ramzan@uws.ac.uk).
Kia Dashtipour is with James Watt School of Engineering,
University of Glasgow, Glasgow, G12 8QQ, U.K. (E-mail:
kia.dashtipour@glasgow.ac.uk).
Kamran Arshad and Khaled Assaleh are with College of Engineer-
ing and IT, Ajman University, Ajman, United Arab Emirates (E-mail:
k.arshad@ajman.ac.ae, k.assaleh@ajman.ac.ae).
negative impact on body human, not caring about correct
posture or fault posture can lead pain in neck, back and
shoulder. Therefore, it is important to control the human
posture to maintain their health and safety during work or
study. Considering the need, the paper reports three major
contributions that are outlined below:
• In this paper, we implemented a novel CNN and LSTM
architecture for automatically identify the posture detec-
tion. It is worth to mention, the deep learning classifiers
unlike machine learning algorithms do not require hand-
crafted features.
• In addition, a novel hybrid approach based on DL (1D-
CNN, 2D-CNN, LSTM, BiLSMT) and ML (random
forest, KNN, Naive Bayes, decision tree, LDA, QDA and
SVM) methods developed to identify the posture.
• There is an extensive comparative experimental results
that are conducted with state-of-the-art approaches to
evaluate the performance of our proposed approach.
This paper discuss the machine learning (ML) and deep
learning (DL) methods for posture detection which is used to
monitor human activity. In this study, we focus on the selection
of ML, DL and hybrid methods to increase the performance
of posture recognition. The activity which is recognised are
sitting and standing. The sitting and standing is important
Authorized licensed use limited to: University of Glasgow. Downloaded on February 03,2021 at 09:11:22 UTC from IEEE Xplore. Restrictions apply.
1530-437X (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2021.3055898, IEEE Sensors
Journal
2 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017
posture to detect because human can monitor their own activity
if they are sitting for long time, they can stand for some
activity.
The paper is organised into the following sections. Section
II provides detailed related work within the field of radar based
motion detection. Section III presents the methodology of how
the experimentation of this research has been done. Section
IV discusses the results obtained through the experimentation.
Section V gives the conclusion.
II. RELATED WORK
In the literature, extensive research has been carried out to
build different posture models. In this section, we summarize
the related state-of-the-art approaches for detection of human
postures.
In smart city prediction and supervision of human health,
using smart technology and portable system is an important
part. Therefore, posture recognition in this paper is determined
with multisensory and using LoRa (Long Range) technology.
LoRa WAN technology has the advantage of long transmission
distances and low cost. Using these two, multisensory and
loRa technology, wearable clothes are designed which is
comfortable in any given posture. In this paper multiprocessing
is used because LoRa has low transmitting frequency and data
transmission size is small. Hence, multiprocessing is done by
sliding window, feature extraction, data processing and feature
selection is done with Random forest. With three testers of 500
grouped data set better performance and accuracy is achieved
[20]. Body postures and gestures are also non-verbal way of
communication. In this paper, Augmented reality is used to
determine the static posture by reducing cost and advanced
body tracking technology. Also unsupervised machine learning
on Kinect body posture sensors are used to detect group
cooperation and learning [21]. Posture detection has also play
an important role in performing yoga more accurately. Posture
recognition is challenging task due to real-time bases and less
available data set. Therefore, to overcome this issue large data
set has been created with at least 5500 images of different
yoga poses. For posture detection tf-pose estimation algorithm
has been used which draws skeleton of human body on the
real-time bases. Angles of the joints in the human body are
extracted using the tf-pose skeleton which is used as a feature
to implement various machine learning models (SVM, KNN,
Logistic Regression, DT, NB and random forest). Among all
Random forest model gives best accuracy [22, 23, 24].
In addition, there is another problem of posture in human
being which occurs due to maximum time in sitting position.
The poor and prolonged sitting effects physical and mental
health. Posture training system is designed for sitting position
and stretch pose data collection. Then for posture recognition,
smart cushion using Artificial Intelligence (AI) and pressure
sensing technologies is used. For more than 13 different pos-
tures, supervised machine learning models are trained which
give better performances [25]. The sensor chair with pressure
sensor tries to avoid wrong sitting position which may cause
disease. In this posture detection, analysis is compared with
decision tree and random forest. The classifier which gives
better performance is random forest classifier [26]. For the
improvement of sitting posture, sitting posture monitoring
systems (SPMSs) is used. It has mounted sensors on backrest
and seat plate of a chair. For this experiment 6 sitting postures
are considered. Then various machine learning algorithms
(SVM with RBF kernel, SVM linear, random forest, QDA,
LDA, NB and DT) are applied on body weight ratio which
is measured by SPMS. Result from SVM with RBF kernel
gives better accuracy as compare to others [27]. There is
also an intelligent systems design for the posture detection
of sitting person on wheel chair. A network of sensors is used
for data collection using neighbourhood rule (CNN), then data
balancing is done with Kennard-stone algorithm and reduction
in dimensions via principal component analysis. Finally k-
nearest algorithm is applied to pre-processed and balanced
data. In this amount of data is significantly reduced but result
is remarkable [28].
A postural habit which has been formed cannot change
easily so it is vital to form a proper postural habit since
childhood. Therefore, machine learning algorithms CNN , NB,
DT, NN, MLR and SVM are used for posture detection. Data
is collected with a sensing cushion which is developed with
(8x8) pressure sensor mat inside children chair seat cushion.
Ten children are participated for five prescribed postures. The
accuracy of CNN is the highest than other algorithms [29].
Dance is also a challenging task for posture recognition. It
is multimedia in nature and its duration is over time as well
as space. For dance analysis few things must be undertaken
like segment of the dance video, recognition of the detected
action element and recognition of the dance sequences. In
this paper focus is on Indian classical dance, Bharatanatyam,
which is driven by music as well as motion for posture
recognition. Then recognition is done by machine and deep
learning techniques which are GMM, SVM and CNN. In the
final step Hidden Markov Model (HMM) is applied for data
sequence recognition. The best recognition rate is with CNN
classifier [30].
III. METHODOLOGY
This section describes our proposed approach for the de-
tection of posture. The Fig shows the overall framework of
posture detection.
A. Feature Extraction
There are total of six features are used for posture predic-
tion. The features determine the posture are skew, percentile,
square root (SR), standard deviation (SD), mean and kurtosis.
The values for each feature is calculated individually for each
window size. For example, the window size of 90 seconds
is selected and the aforementioned features are calculated.
After the feature extraction, the new dataset is created which
consists of different features. It is to be noted that, after
feature extraction, the most important task is to determines
the combination of best features for posture prediction in
term of accuracy. In total six features and combination of
these features are evaluated using different ML models. In
total combination of features for posture prediction is time
Authorized licensed use limited to: University of Glasgow. Downloaded on February 03,2021 at 09:11:22 UTC from IEEE Xplore. Restrictions apply.
1530-437X (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2021.3055898, IEEE Sensors
Journal
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE TRANSACTIONS AND JOURNALS (FEBRUARY 2017) 3
TABLE I
MACHINE LEARNING METHODS WITH THEIR PARAMETERS
Algorithm Parameters Training time
KNN Eculidean distance 2 m 30 s
SVM RBF Kernal 3 m 41 s
Random Forest Max depth = 2 3 m 27 s
K-nearest neighbors 3 3 m 2 s
Logistic regression Penalty = l2 3 m 41 s
QDA tol = 0.0001 2 m 31 s
Naive Bayes sample weight = none 2 m 48 s
LDA number of components = 5 3 m 28 s
LSTM 10-layered 10 m 23 s
BiLSTM Adam optimizer 7 m 13 s
1D-CNN dropout = 0.2 9 m 16 s
2D-CNN dropout = 0.2 8 m 2 s
consuming therefore, we employed DL methods which dp
not require any feature engineering and they usually obtain
superior performance as compared to ML models.
B. Machine Learning (ML) Methods
After feature extraction, there are different machine learning
classifiers, including SVM, logistic regression, KNN, decision
tree, naive bayes, random forest, LDA and QDA have been
applied in order to evaluate the performance of the approach.
Table I shows the parameters that are used to trained the
machine learning methods. The scikit-learn package is used to
train the machine learning classifiers. In addition, the training
time for each models has been presented in Table I.
C. Deep Learning (DL) Methods
In next section, we discuss our proposed hybrid model
which integrates the machine learning and DNN methods
including 1D-CNN, 2D-CNN, LSTM and BiLSTM. The deep
learning can be used in different various application such as
cyber-security, sentiment analysis, speech enhancement and
etc. [31, 32, 33, 34, 35, 36, 37, 38] . However, in this paper
we proposed a novel framework to detect posture prediction.
Convolutional Neural Network: For comparison, the novel
CNN framework is developed. The implemented CNN consists
of input, hidden and output layers. Our proposed CNN frame-
work contains convolutional, max pooling and fully connected
layers. The 10-layered CNN framework achieved the most
promising results. The parameters of CNN framework are
shown in Table II
Long Short Term Memory (LSTM): The long short term
memory (LSTM) proposed architecture contains input layer,
two different stacked LSTM and one output as fully connected
layer. Particularly, the LSTM architecture consists of two
different stacked bidirectional layers (contains 128 cells and
64 cells) with dropout 0.2 and a dense layer with two neurons
and softmax activation.
D. Hybrid Models
The hybrid methods consists of different classifiers and
combining their prediction to train meta-learning model. The
hybrid is used to enhance the performance of specific sys-
tem. In this study, the prediction of ML classifiers (logistic
regression, random forest, KNN, Na
¨
ıve Bayes, decision tree,
linear discriminant analysis, quadratic discriminant analysis
and SVM) and DL classifiers (CNN, LSTM) are used as input
of CNN, LSTM architecture. Fig 2 shows the architecture of
proposed hybrid of ML and DL for posture detection. It is to
be noted that, the parameters of each classifier has been set-up
empirically after several simulation experiments.
IV. EXPERIMENTAL RESULTS
In order to classify the posture prediction, standard (logistic
regression, random forest, KNN, Na
¨
ıve Bayes, decision tree,
linear discriminant analysis, quadratic discriminant analysis
and SVM) and deep learning classifiers such as (1D-CNN,
2D-CNN, LSTM, BiLSTM) are trained. We extracted differ-
ent features including skew, percentile, SR, SD, mean and
kurtosis. It is worth to mention that, there are total thirteen
experiments have been done. In addition, the 10-fold cross-
validation is used to perform the experiments. In order to
evaluate the performance of the proposed approach, precision,
recall, F-score and accuracy metrics were used:
P recision =
T P
T P + F P
(1)
Recall =
T P
T P + F N
(2)
F measure = 2 ∗
P recision ∗ R ecall
P recision + R ecall
(3)
Accuracy =
T P + T N
T P + T N + F P + F N
(4)
Human Body Posture Detection: In order to evaluate the
performance of the approach. The online widely benchmark
dataset called human body using galvanic skin response have
been used. The data is collected for five different subjects, and
it has been classified into three different categories such as
standing, sitting and walking. In addition, the data is recorded
at a resolution of 16 bits in samples of 5 min to 15 min and
the sampling rate is 1 MHz (maximum precision position on
the BITalino Kit) [39]. There are different machine learning
methods to train the classifiers for classifying the posture
such as standing, sitting and walking. In order to evaluate
the performance of the approach, the data used in our study is
collected from five different individuals, the dataset consists of
four males and females for different ethnicity, all the bracket
of 25 to 30 years of age.
The machine learning algorithms are trained based on the
10-fold cross-validation and train/test used Python variables
containing the data and comparing the prediction of the data
to the actual labels of the data. There are different evaluation
metrics such as accuracy, precision, recall and f-measure are
used to compare the current algorithms. It is worth to mention
that, the scikit-learn with Tensorflow background is used to
implement the deep learning approaches.
Table III shows the summary of results for selected features
using logistic regression. The experimental results show the
combination of all features achieved better performance and
mean feature achieved less accuracy.
Authorized licensed use limited to: University of Glasgow. Downloaded on February 03,2021 at 09:11:22 UTC from IEEE Xplore. Restrictions apply.
1530-437X (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2021.3055898, IEEE Sensors
Journal
4 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017
TABLE II
CNN ARCHITECTURE (CONV - CONVOLUTIONAL LAYER, MAXPOOL - MAXPOOLING LAYER, GLOBALMAXPOOL - GLOBAL MAX POOLING LAYER,
FC - FULLY CONNECTED LAYER, RELU - RECTIFIED LINEAR UNIT ACTIVATION
Layer 1 2 3 4 5 6 7 8 9 10
Type Conv Max Conv Max Conv Max Conv Global Fc Fc
Filters 16 Pool 32 Pool 64 128 Max
K
ernal Size
3 2 3 2 3 2 3 Pool
Neurons 128 2
Activation ReLU ReLU ReLU ReLU ReLU SoftMax
Fig. 1. Overview of the proposed framework for posture detection
Fig. 2. Proposed Hybrid Classifier
Table IV shows the summary of results for selected features
using random forest. The experimental results show the com-
bination of all features achieved better performance. In the
other hand, mean and SD feature achieved less accuracy.
Table V shows the summary of results for selected features
using KNN. The experimental results show the combination
of all features achieved better performance. In the other hand,
mean feature achieved less accuracy.
Table VI shows the summary of results for selected features
using Naive Bayes. The experimental results show the mean
and SD and also mean, SD and SR features achieved better
performance. In the other hand, mean, SD, SR and percentile
features achieved less accuracy.
Table VII shows the summary of results for selected fea-
tures using decision tree. The experimental results show the
combination of all features achieved better performance. In the
other hand, mean and SD features achieved less accuracy.
Table VIII shows the summary of results for selected
features using LDA. The experimental results show the mean,
SD, SR and percentile features achieved better performance.
Authorized licensed use limited to: University of Glasgow. Downloaded on February 03,2021 at 09:11:22 UTC from IEEE Xplore. Restrictions apply.
