JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1
Learning-Assisted Optimization in Mobile Crowd
Sensing: A Survey
Jiangtao Wang, Yasha Wang, Daqing Zhang, Jorge Goncalves, Denzil Ferreira, Aku Visuri, Sen Ma
Abstract—Mobile Crowd Sensing (MCS) is a relatively new paradigm for collecting real-time and location-dependent urban sensing
data. Given its applications, it is crucial to optimize the MCS process with the objective of maximizing the sensing quality and
minimizing the sensing cost. While earlier studies mainly tackle this issue by designing different combinatorial optimization algorithms,
there is a new trend to further optimize MCS by integrating learning techniques to extract knowledge, such as participants’ behavioral
patterns or sensing data correlation. In this article, we perform an extensive literature review of learning-assisted optimization
approaches in MCS. Specifically, from the perspective of the participant and the task, we organize the existing work into a conceptual
framework, present different learning and optimization methods, and describe their evaluation. Furthermore, we discuss how different
techniques can be combined to form a complete solution. In the end, we point out existing limitations which can inform and guide future
research directions.
Index Terms—Mobile Crowd Sensing, Learning, Optimization.
F
1 INTRODUCTION
C
OINED by Howe and Robinson in [1], the idea of
crowdsourcing has become an emerging distributed
problem-solving paradigm by combining the power of both
human computation and machine intelligence. Furthermore,
the prevalence of mobile devices and the increasing smart
sensing requirements in the city have led to an alternative or
complementary approach for urban sensing, called Mobile
Crowd Sensing (MCS) [2], [5]. MCS leverages the inherent
mobility of mobile users (i.e., participants or workers),
the sensors embedded in mobile phones and the existing
communication infrastructures (Wi-Fi, 4G/5G networks) to
collect and transfer urban sensing data. Compared to wire-
less sensor networks (WSN), which are based on specialized
sensing infrastructures, MCS is less costly and can obtain a
higher spatial-temporal coverage.
However, every coin has its two sides. Although with
the above advantages and various MCS-enabled innovative
applications [8], [9], [10], [11], [12], [13], the new sensing
paradigm also encounters new challenges as ”humans” act
as sensors [14]. First, the sensing quality problem is more
complex in MCS, because human sensors are quite complex
and several human factors have to be taken into account. For
example, it is uncertain to predict if the participants would
accept the recommended sensing tasks or not. Even if they
accept the task, factors such as reliability, user preference,
expertise, and mobility pattern may significantly affect how
they will complete these tasks (e.g., coverage and sensing
• Jiangtao Wang and Daqing Zhang are with school of EECS, Peking
University. Yasha Wang and Sen Ma are with National Research &
Engineering Center of Software Engineering in Peking University. Jiang-
tao Wang, Yasha Wang, Daqing Zhang and Sen Ma are also with
Key Laboratory of High Confidence Software Technologies, Ministry of
Education. Jorge Goncalves is with School of Computing and Information
Systems, University of Melbourne, Australia. Denzil Ferreira and Aku
Visuri are with University of Oulu, Finland.
• Jiangtao Wang and Yasha Wang are the corresponding authors.
• This work was mainly supported by NSFC Grant (No. 61872010).
quality). Second, participating in an MCS campaign incurs
extra cost (e.g., energy consumption and data transferring
cost) and concerns (e.g., location privacy leak) to the par-
ticipants. Keeping the cost as low as possible is beneficial
for motivating participants to contribute their data. In sum-
mary, with the objective of maximizing sensing quality and
minimizing sensing cost control, it is crucial to optimize the
entire lifecycle of MCS, and the number of relevant research
works has continuously increased in recent years.
Earlier studies mainly tackle this issue from the per-
spective of designing different combinatorial optimization
algorithms in participant selection or task assignment. With
the rapid technical progress in learning-based artificial in-
telligence, we notice that it is now an emerging trend to
integrate the learning techniques into the research problem
of MCS optimization. On the one hand, a group of studies,
such as [21], [22], [23], [25], [26], [27], focus on how to
understand participation behavior, and then exploit the ob-
tained knowledge to future optimize the MCS process (such
as participant selection and task assignment). On the other
hand, another category of works, such as [42], [43], [44], [48],
[49], [50], leverage the correlation among sensing data (such
as spatial-temporal correlation) or data inference techniques
to optimize MCS in several aspects, such as reducing the
cost in sensing data sampling and discovering truth through
sensing results aggregation.
In recent years, there are several survey or tutorial
papers in the MCS research community. Some [2], [4], [5]
focus on the description of overall and general picture (e.g.,
lifecycles, research issues, and challenges) in MCS, and
others such as [15], [16], [17], [18], [19] dive into specific
research topics in MCS, including incentive mechanisms
[15], [16], privacy preservation [17], [18], and energy saving
[19]. However, to the best of our knowledge, there are
no survey or tutorial papers summarizing how learning
techniques are explored to assist the MCS optimization
process. Therefore, this motivates the need for a compre-
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 2
hensive survey.
With the above motivation, we conduct a comprehensive
survey of all publications related to learning-assisted MCS
optimization via a paper selection process guided by a
suggestion made in [3]. The main criteria for including a
paper are: a) whether it describes a research problem in
MCS or similar concepts (e.g., participatory sensing, mobile
crowdsourcing, and spatial crowdsourcing), and b) does the
article utilize learning techniques to optimize a certain as-
pect of MCS. We performed three types of literature searches
before Nov 2017: a) Online digital libraries including ACM,
IEEE Xplore, Springer Link, Wiley, Elsevier ScienceDirect,
and Google Scholar. b) Main conference proceedings and
journals in fields such as ubiquitous computing, mobile
computing, and wireless sensor networks from January
2008 to Nov 2017. The specific conference proceedings and
journals are on the top proceeding lists within the fields
[67]. c) By searching the citations from included papers, we
further discovered some additional relevant papers.
The contributions of this survey paper include:
1) We present a comprehensive survey of the literature
using learning techniques to optimize the process of MCS,
which is a hot topic in MCS research community but lacks
survey or tutorial papers. To the best of our knowledge,
this article is the first work summarizing MCS optimization
techniques from a learning-assisted perspective.
2) We classify the relevant works from the perspective
of both participants and tasks, with the objective of maxi-
mizing quality or minimizing cost. In addition to presenting
each individual technique, we discuss how they are evalu-
ated, analyze their relationships, and discuss how they can
be combined to optimize MCS systems collaboratively.
3) We highlight the existing gaps for the state-of-the-art
learning-assisted MCS optimization approaches and present
some future research opportunities.
2 MCS AND ITS OPTIMIZATION
In this section, we present some basic background knowl-
edge about MCS and its optimization. For more detailed
understanding about MCS and its main research issues,
interested readers can refer to other surveys and tutorials
[2], [4], [5].
2.1 Preliminary of MCS
Compared to general crowdsourcing, MCS have two unique
features. (1) Mobility-Relevant Features. Different from gen-
eral crowdsourcing tasks, MCS requires the workers to com-
plete sensing tasks in certain locations, because the sensing
results are location-dependent (e.g., air quality, noise level,
and traffic congestion status). (2) Sensing-Relevant features.
Different from general crowdsourcing, MCS always targets
at urban sensing tasks. First, the execution of sensors and
localization modules introduces much more energy con-
sumption into MCS than general crowdsourcing. Therefore,
it is important to control the energy consumption of workers
in the MCS systems. Second, many MCS tasks need to
invoke phone-embedded sensors for task completion, but
the set of sensors for each worker may be different as they
hold various brands and models of smart devices.
Similar to the notion of participatory sensing [6] and
human-centric computing [7], there are two key players in
MCS, i.e., participants who collect and report sensing data
through a mobile device, and task organizers who manage
and coordinate the whole MCS process. The life-cycle of
MCS can be divided into four stages: task creation, task
assignment, task execution and data aggregation. The main
functionality and research issues of each stage are briefly
described as follows:
a) Task Creation: The MCS organizer creates an MCS
task to be given to workers with the corresponding mobile
applications. In this stage, the key research issue is how to
reduce the time and the technical threshold of task creation
[62], [63].
b) Task Assignment: After the organizer creates an MCS
task, the next stage is task assignment, in which the MCS
platform selects participants and assigns them with the
different sensing tasks. The key research issue at this stage
is how to optimize MCS taking into account a number of
different factors, such as spatial coverage, incentive cost,
energy consumption, and task completion time [29], [64].
c) Task Execution: Once the participants have received
the assigned micro-sensing tasks, they can complete them
within a pre-defined spatial-temporal scale (i.e., time dura-
tion and target region). This state includes sensing, comput-
ing, and data uploading. How to save energy consumption
and protect users’ location and overall privacy are the core
research challenges at this stage [18], [19].
d) Data Integration: This stage fuses the reported data
from the crowd according to the requirements of task orga-
nizers. The key issue at this stage is how to infer missing
data and provide a complete spatial-temporal picture of the
target phenomenon (e.g., real-time air quality map of a city)
[43], [45].
2.2 Two Aspects of MCS Optimization: Quality and
Cost
For the optimization of MCS, the control of sensing quality
and cost is a fundamental research problem. On the one
hand, we want to maximize the sensing quality of an MCS
task. The sensing quality metric can be diverse for differ-
ent applications (e.g., spatial-temporal coverage, Quality-of-
Information, mean error rate, etc.) [20]. On the other hand,
we need to control the cost during the MCS process. The cost
may include incentive rewards, energy consumption, data
transferring expense, privacy leak, attention occupation, etc.
[15], [16], [17], [18], [19].
However, the sensing quality maximization and sensing
cost minimization are usually two opposing objectives. For
example, to optimize the spatial-temporal coverage, we may
need to recruit more participants, which will then lead to a
higher total sensing cost. Therefore, how to achieve a good
tradeoff between sensing quality and cost is a major research
issue in MCS.
3 LEARNING-BASED MCS OPTIMIZATION: A
CONCEPTUAL FRAMEWORK
In Fig 1, we present a conceptual framework for learning-
based MCS optimization, which primarily consists of the
following two phases.
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 3
Fig. 1: Conceptual Framework for Learning-Based MCS
Optimization
Learning Phase: we can extract knowledge from both
participants and tasks. In terms of participants, with various
machine learning techniques (such as classification, cluster-
ing, and regression), we can form a better understanding
towards both the individual or the community of the par-
ticipants for several aspects, such as willingness, mobility
pattern, sensing context, ability, and reputation. In terms
of tasks, it analyzes and discovers the correlation between
different types of sensing data and tasks.
Optimization Phase: we can leverage the extracted
knowledge to optimize the MCS campaign in the following
aspects:
1) Sensing quality control. MCS faces the challenge of low-
quality or even erroneous data collection. For example, a
smartphone may report inaccurate data samples when it is
located in a bag or pocket [21], or a participant may report
malicious sensing data for his own benefit [2]. With this
in mind, we need to integrate the extracted knowledge to
enable quality-optimized MCS. For example, in both the
participant selection and ground truth inference phase, we
should assign a higher priority to the participants who are
more willing to accept tasks, more reliable, and with better
spatial-temporal coverage.
2) Sensing cost control. The process of participation in
MCS campaigns leads to costs such as energy consumption,
data transferring fee, and attention occupation for the par-
ticipants. To compensate these, the task organizer needs to
pay incentive rewards to motivate a large number of partic-
ipants. The extracted knowledge in the learning phase can
help us reduce cost. For example, with spatial correlation in
mind, we can select a subset of more informative areas (i.e.,
having the highest information gain in terms of deducing
the sensing data in other unselected areas), and then deduce
the sensing data in unselected ones.
The above process is iterative in nature, in which the
behavior data about participants and sensing data of MCS
tasks are collected continuously to update the multi-aspect
knowledge. Then, the updated knowledge will be further
used in the MCS process.
4 LEARNING AND OPTIMIZATION TECHNIQUES
In this section, we present the existing approaches for opti-
mizing MCS through learning techniques and summarize
their contributions. We divide the state-of-the-art studies
into the following groups: (1) Participant-Oriented Learning:
based on the participants’ profile, historical mobility traces,
and participation records, we can learn and predict partici-
pants’ behavior in MCS, which can be leveraged to recruit
and select more beneficial participants, or assist them to
better complete sensing tasks. (2) Task-Oriented Learning: the
objective of this group of research works is to mine the data
correlation in MCS tasks, and then exploit this to reduce the
sensing cost or improve sensing quality.
4.1 Participant-Oriented Learning
A number of research studies use a data-driven approach
to learn participants’ behavioral patterns and exploit it in
assigning tasks to more preferred participants. As a given
study may involve several aspects, Table 1 summarizes this
set of works in terms of the learned knowledge.
1) Willingness. Most of existing works (such as [29], [30],
[31], [32], [33]) assume that once a participant is assigned
with a task, she/he will accept and complete it. However,
this is not true in real-world settings, as participants may re-
ject the task due to several reasons. Neglecting this issue has
negative impact on the performance of MCS applications.
To address this problem, the authors in [22] conducted a 4-
week extensive smartphone user study to explore what are
the factors influencing participants’ participation willing-
ness. Their findings show that data was shared significantly
more when anonymously collected, and that the data type
is also an important factor. The authors in [28] carried
out a study in Chicago to explore the geographic factors
influencing the participation willingness, and quantitative
modeling shows that travel distance to the location of the
task and the socioeconomic status (SES) (i.e. a measure
of ones’ economic and social position based on income,
education, and occupation) of the task area are important
factors. These results indicate that low-SES areas are cur-
rently less able to take advantage of the benefits of MCS. In
a mobile crowdsourcing framework named GP-Selector [23],
the authors developed a multi-classifier based approach to
infer if a participant will accept an MCS task or not, where
the influencing features are the incentive reward, domain
interest, task workload, and privacy concern. In the focused
scenario of [24], the authors assume that the participants
decide whether to accept the task based on the incentive
reward and movement distance. They developed a SVM-
based method to learn the relationship between task accep-
tance rate and these two factors, and then utilize it to design
better pricing mechanisms, with the objective of reducing
sensing cost while ensuring task completion. In [25], the
authors have taken participants’ rejection into consideration
and tried to maximize the overall acceptance in order to
improve the system throughput. Lastly, whether a person
can be interrupted in a given situation also influences the
likelihood of willingness, as explored in [68], especially if
the contribution relies on manual reporting.
2) Mobility Pattern. Contrary to generic online crowd-
sourcing, MCS requires the participants’ physical movement
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 4
to specific locations for task completion. Thus, the mobil-
ity pattern of the participants significantly affects the task
assignment process. In [26], [27], based on a real-world
deployed MCS platform in campus, authors provided an
analysis for the efficiency of recommending tasks based on
predicted movement patterns of individual workers. With
the goal of optimizing the spatial-temporal coverage in
budget-constrained MCS, a group of works such as [29],
[30], [31], [32], [33] studied the optimal task allocation based
on the learning participants’ mobility pattern from the pre-
vious trajectories. For example, [29], [31], [32], [33] assumed
that the number of calls in each spatial-temporal cell follows
a Poisson distribution, and they calculate the probability of
participants’ presence in each spatial-temporal cell based
on historical trajectories. The authors in [30] adopted a
location probability transition approach (i.e., calculating the
transition probability between two locations) to accomplish
mobility learning and prediction.
3) Sensing Context. Sensing context (e.g., the partici-
pants’ motion and the position of the mobile device) has
a significant impact on the sensing data quality for certain
types of MCS tasks. The authors in [34] trained a sensing
data quality classifier, which extract the relation between
context information (such as the participants’ motion) and
sensing data quality, to estimate data quality in MCS. This
classifier can be applied to guide user recruitment and task
assignment in MCS.
4) Ability and Reputation. Learning participants’ abilities
and reputations can help selecting more capable and reliable
participants [35], [36], [37], [38], [39]. For instance, through
an empirical study, [35] revealed that participants’ cognitive
abilities correlate tightly with their crowdsourcing perfor-
mance, where they built two models for crowdsourcing task
performance prediction. In another example, [36] proposed
a reputation-based system that employs the Gompertz func-
tion for learning the participants’ reputation score, and
implement this idea in the scenario of a crowd noise level
monitoring application. Though with different definitions of
reputation metrics, they learn the reputation scores in either
of the two categories: 1) statistical reputation scores that are
computed based on the comparison between reported data
and estimated the ground truth. 2) vote-based reputation
scores by the participants of MCS.
4.2 Task-Oriented Learning Approaches
Learning techniques also can be used to extract knowledge
from the perspective of the tasks. Here we will present how
the learning approach can optimize MCS in sensing data
correlation learning and sensing data aggregation.
4.2.1 Sensing Data Correlation Learning
Learning and exploiting sensing data correlation is an im-
portant technique to optimize the MCS process. It is based
on the notion that, typically, there is a correlation among
diverse sensing targets in the real world, and we can use
this to address the sensing data redundancy and sparsity
issues in MCS. By appropriately using data correlation, we
can require the participants to collect only a relatively small
number of data samples and deduce more information, thus
the cost of MCS is significantly reduced.
In recent years, a number of studies in MCS focus
on these aspects. Both [40] and [41] investigated a traffic
status monitoring task, in which they use the correlation
between the traveling speed on different roads sections to
maximize the sensing accuracy with a fixed number of
crowd sensors. The authors in [42], [43], [44], [45] utilized
the spatial-temporal correlation of environmental sensing
data (e.g., temperature and air quality) to achieve an op-
timized tradeoff between sensing cost and quality, in which
they use matrix completion technology to infer the missing
sensing data. The study in [46] demonstrated the feasibility
of applying compressive sensing to data domains like large-
scale question-based user surveys. The approach proposed
in [47] is the extension of [46], which considered the sensing
data reliability in different subareas due to different sam-
pling density. Both [48] and [49] built a dependency graph
between different entities in the city (such as the availability
of shops and gas stations) to increase fact-finding accuracy.
Focusing on the scenario where MCS is utilized to collect
training data of context-aware applications, [50] proposed
an active learning framework for optimally budgeted MCS.
The authors in [73] exploited the spatial-temporal correla-
tion of users’ mobility to achieve the tradeoff between MCS
task performance and privacy preserving objective.
Although the above literature is different in terms of data
type and detailed algorithms, they also attempt to address
one of the three important issues: 1) Informative Sampling:
how to select the most informative data collections? 2)
Missing Data Inference: how to infer the missing data from
the obtained one? 3) Quality Estimation: how to estimate
if the inference meets the accuracy requirement without
ground-truth sensing data. The summary is in Table 2.
4.2.2 Sensing Results Aggregation
Different from the traditional wireless sensor network, MCS
faces the challenge of unreliable data samples due to many
reasons (e.g., uncertain sensing context and malicious par-
ticipants). To achieve high-quality results, we need to col-
lect sensing data from multiple participants for the same
sensing target and infer the truth. This problem is similar
to truth discovery, which has been studied extensively in
the general crowdsourcing community. Specifically, there
are two inputs, i.e., the task answers and the expertise of
each participant. Recently, a survey has comprehensively
summarized this topic [51], where most of the literature [52],
[53] use voting-based strategies, such as majority voting,
weighted voting, Bayesian voting, etc.
Different from general online crowdsourcing, the truth
discovery problem in MCS is more complex because of
the multi-modality nature and spatial-temporal features of
the sensing data, and some participant-side factors (e.g.,
location privacy). Thus, the techniques that existing works
adopted for truth discovery in MCS are different to some
degree. A number of works [54], [55], [56], [57], [58] lever-
aged Expectation Maximization (EM) based algorithms to
estimate the reliability of participants or mobile devices,
which will be used as the weight to infer the ground truth
of sensing data. Some other works [59], [60] have adopted
unsupervised learning approaches, in which they employ an
additional optimization objective to improve the EM-based
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 5
TABLE 1: Learning participant-side factors to optimize MCS: a summary
References Willingness Mobility Sensing context Ability Reputation
[22] Yes
[23] Yes Yes Yes Yes
[24] Yes Yes
[28] Yes Yes
[29], [30], [31], [32], [33] Yes
[34] Yes Yes
[35] Yes
[36], [37], [38], [39] Yes
TABLE 2: A summary of studies to optimize MCS through the learning of the data correlation
Literatures Data Type Informative Sampling Missing Data Inference Quality Estimation
[40] Traffic speed Heuristic greedy Markov random field Not addressed
[41] Traffic speed Not addressed Matrix completion Not addressed
[42], [43] Temperature, air quality
The variance of
different inference algorithm
Matrix completion Leave-one-out estimation
[44] Air quality Not addressed Matrix completion Not addressed
[45] Temperature, air quality, traffic speed
The variance of
different inference algorithm
Matrix completion Leave-one-out estimation
[46] Question-based user surveys Compressive sensing Matrix completion Not addressed
[48], [49] Availability of urban entities Not addressed Bayesian Network Not addressed
[50]
Labels and training data
for activity recognition apps
Active Learning Not addressed Not addressed
method. Recently, truth discovery concerning the privacy-
preserving issue has been studied [61], which infers the
missing data using matrix factorization techniques.
5 HOW TO CONDUCT EVALUATION
One important question about the research on learning-
based MCS optimization is that: Where can we get the training
data, and how to evaluate the performance of a given approach?
We know that the ideal way is to obtain large-scale data
about participants’ behavior and collected sensing data,
based on which extensive evaluation can be conducted.
However, it is difficult to conduct such a large-scale and
real-world evaluation as platforms, such as Amazon Me-
chanical Turk and WAZE, are not willing to open their
data due to commercial reasons. Thus, researchers adopt
alternative ways to demonstrate the feasibility of their pro-
posed approach. In this section, we summarize different
methodologies, which we hope can inspire and support the
evaluation of future research efforts.
By summarizing the existing work, the evaluation
methodology can be divided into the following three cat-
egories.
1) Small-scale real-world evaluation. A group of studies
develops their own testbed to collect relevant data for eval-
uation. For example, the authors in [26], [27] build campus-
scale MCS platforms as the research testbeds, in which 80
real users are recruited to complete several types of MCS
tasks within a 4-week period. Similar platforms such as
gMission and ChinaCrowds are developed and utilized in
studies such as [25], [56], [69], [70].
2) Open dataset based evaluation. Another group of re-
search works evaluates their solutions based on an open
datasets (such as D4D
1
, Gowalla
2
). For example, [29], [31],
[32], [33], [71] evaluate a mobility pattern learning algorithm
1. http://www.d4d.orange.com/en/presentation/data
2. https://snap.stanford.edu/data/loc-gowalla.html
and task assignment approach based on open data contain-
ing the mobility trace of a large number of participants
(e.g., calling trace and check-in data in a social network).
Furthermore, [48], [49] evaluate their dependency analysis
approach with a real-world dataset about the availability
of groceries, pharmacies, and gas stations during Hurricane
Sandy. The authors in [42], [43] evaluate their missing data
inference algorithms based on a campus-scale open dataset
for temperature and air quality measures.
3) Simulation-based evaluation. Another alternative way
is to develop a simulator, in which the agents (both the
participants and task organizers) are simulated according
to pre-defined rules. Then, we can use the simulated data
generated by the agents to perform the evaluation. A sig-
nificant number of studies adopt the simulation-based ap-
proach to evaluate their learning-based MCS optimization
approaches [23], [25], [27], [29], [31], [32]. We also note that
several papers published in top venues choose to conduct an
evaluation of both the real-world and simulated data. This
is because real-world data is always better, but they often
constitute isolated points in a large space. The simulation, in
contrast, can extensively test the performance under various
settings. Conducting the experiments based on both these
two types of data can make the research work more solid.
Actually, we believe that a promising method should
be the combination of both real-world and simulative eval-
uation. For example, we can collect small-scale and real-
world data to generate some key parameters, and use these
parameters to enable a large-scale simulation. For example,
in [66], the authors learn the distribution of the parameters
about the participants’ preferences in completing MCS tasks
using real-world data from 80 participants during 4 weeks.
Then, they further evaluate the proposed algorithm by a
simulation study, in which the parameters are generated
based on the pre-learned distribution.
