1
The Power of Prediction with Social Media
● Harald Schoen, University of Bamberg (Germany), harald.schoen@uni-bamberg.de
● Daniel Gayo-Avello, University of Oviedo (Spain), dani@uniovi.es
● Panagiotis Takis Metaxas, Wellesley College and Harvard University (USA), pmetaxas@seas.harvard.edu
● Eni Mustafaraj, Wellesley College (USA), eni.mustafaraj@wellesley.edu
● Markus Strohmaier, Graz University of Technology (Austria), markus.strohmaier@tugraz.at
● Peter Gloor, MIT (USA), pgloor@mit.edu
Abstract
Social media today provide an impressive amount of data about users and their societal
interactions, thereby offering computer and social scientists, economists, and statisticians –
among others– many new opportunities for research exploration. Arguably, one of the most
interesting lines of work is that of predicting future events and developments based on social
media data, as we have recently seen in the areas of politics, finance, entertainment, market
demands, health, etc. In fact, an average of one in seven research papers presented at the WWW,
ICWSM and IEEE SocialCom Conferences between 2007 and 2012 contain the term “predict” in
their title. This upward trend, starting from 0 in 2006 and reaching 18% in 2012, shows a
significant interest of the research community in predicting with Social Media.
But what can be successfully predicted and why? Since the first algorithms and techniques
emerged rather recently, little is known about their overall potential, limitations and general
applicability to different domains.
Better understanding the predictive power and limitations of social media is therefore of utmost
importance, in order to be successful and avoid false expectations, misinformation or unintended
consequences. Today, current methods and techniques are far from being well understood, and it
is mostly unclear to what extent or under what conditions the different methods for prediction
can be applied to social media. While there exists a respectable and growing amount of literature
in this area, current work is fragmented, characterized by a lack of commonly accepted
evaluation approaches. Yet, this research seems to have reached a sufficient level of interest and
relevance to justify a dedicated section.
This special section aims to shape a frame of important questions to be addressed in this field,
and fill the gaps in current research with presentations of early research on algorithms,
techniques, methods and empirical studies aimed at the prediction of future or current events
based on user-generated content in social media.
2
Introduction
Human beings are fascinated with what will happen in the future and, indeed, we even associate
intelligence with an ability to predict future events (Hawkins 2004). In ancient times, several
techniques were invented including inspecting bird flights, haruspicy, and astrology. Later,
predictions were done mostly through experts who had developed their own intuitions and
methods of prediction. Unfortunately, such expert knowledge is idiosyncratic and cannot be
automatized or even duplicated. In more recent times, the research community has developed
much more sophisticated techniques that aim to predict future outcomes using data-based models.
Such model-based forecasts have proved to be quite successful in predicting a diversity of
outcomes including economic, societal, and political outcomes (e.g., Campbell 2008; Clements
and Hendry, 2011; Silver 2012). Despite their general success, even these models cannot predict
the future perfectly, because real-world outcomes can change in ways that are not anticipated by
data-based models.
The advent of social media provides researchers with a new and rich source of easily accessible
data about individuals, society and, potentially, the world in general. In particular, data from
social media captures online behavior of users who communicate or interact on a diversity of
issues and topics. It is the intent of this special section to focus on novel methods of prediction
that are based on data harvested from social media. In recent years, such data has shown to be
very popular with scholars interested in developing predictive models. With varying success, an
emerging community of researchers has utilized social media data for a wide variety of purposes,
for example, to predict stock market movements (e.g., Bollen et al., 2010), to predict
announcements of flu outbreaks (Lampos et.al., 2010), to forecast box-office revenues for
movies (Asur and Huberman, 2010) and even to predict election outcomes (Gayo-Avello, 2012),
to name a few. The models and areas of application are diverse and, moreover, predictions based
on social media data have also attracted considerable attention from the public through
traditional and online media. These media are projecting an impression of social media as a
widely accepted and reliable source of data for predicting future outcomes.
However, reality is more complicated than that. There are many theoretical and methodological
issues in predicting future outcomes using social media data that are far from being settled, and
deeper studies and experiments are required to discover the true potential of social media as a
reliable source of data. While prediction represents a problem in a wide variety of scholarly
fields, social media-based forecasts today receive significant attention. We thus consider it
appropriate to discuss and reflect on the promises of social media based forecasts as well as the
perils and pitfalls it is plagued with, and strategies to address these problems.
This special section aims to serve as a platform for these and related matters. Its intended
audience is computer scientists, social scientists, economists, statisticians, and other researchers
interested in the application of multidisciplinary approaches to exploit user-generated contents to
better understand (and predict) societal behaviors. This issue includes three works approaching
3
such topics from different points of view: the credibility of data appearing in social media; the
detection of unexpected phenomena as deviations of the “pulse” of social media; and a
conceptual framework to survey the current body of research.
This guest editorial is organized as follows: First, we discuss the different approaches to build
forecasting models. Then, we analyze how such models could be adapted to the special
circumstances of social media and the caveats that apply (e.g., the pervasive need of machine
learning methods to ensure the quality of data). Additionally, we discuss one peculiar
idiosyncrasy of social media-based forecasting: the fact that sometimes it is not forecasting but
nowcasting; i.e. the variable of interest is estimated in real time using online trails as proxies.
After that, we briefly survey the most representative research conducted in the topic up to date,
and introduce the papers published in this special section. The editorial concludes with some
final remarks.
Different Types of Forecasting Models
A fundamental question we need to address in order to tackle with previous issues is: What
enables prediction based on social media data? A first requirement is that the prediction itself
somehow must be encoded within the data; without any signal w.r.t. the phenomenon of interest
the data would be rendered useless. Second, the data collection needs to maintain the encoding of
the answer. Third, the analysis performed on the collected data is able to reveal the prediction.
Without all three of these fundamental requirements, predictions are either not possible or no
better than pure chance.
It is fundamental, therefore, to examine the ways the research community conducts both the
process of collecting data and its analysis. We observe that there are three prevailing practices:
Data could be collected through past logs of experiences, and statistical models are often
employed to make sense of them. Data could also be collected on demand. A traditional and
direct way to do that is by using polling, asking the public directly for their opinion or behavioral
intentions, as is done with survey models. However, social media provides an additional and
indirect
1
way to collect data on demand. Researchers can unobtrusively approach social media to
observe the public’s behavior and then derive their intention or opinion from the observed
behavior (e.g., using machine-learning techniques, cf. Bishop, 2006). When the interest lies in
the users’ opinion about the outcome of an event, rather than their intention with regards to it, the
method is somewhat comparable to prediction markets models.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1
Maybe it would be more appropriate to denote such kind of collection as unobtrusive or non-
reactive; see, for instance, Janetzko (2008) for a broad introduction to this approach to
measurement in social sciences.
1
4
In the following subsections we will discuss these different types of forecasting models we have
seen in various fields. Without loss of generality, we use electoral predictions as a running
example for the discussion.
Prediction Market Models
The prediction market model attempts to capitalize on the so-called “wisdom of crowds”
approach (Surowiecki 2004). A large number of people give their best guesses for an outcome
variable. In this respect, this approach is based on subjective evidence. Then, the individual
guesses are aggregated in some way and the aggregate guess, according to this line of reasoning,
will closely approximate the real outcome. This approach is underlying a host of prediction
markets (e.g., Arrow et al., 2008; Rhode and Strumpf, 2004; Forsythe et al., 1992). Participants
deal with assets that are linked to the quantity of interest, i.e. the occurrence of an outcome or a
parameter, such as a party’s vote share. Market prices are thus interpreted as predictions of the
occurrence probability or another parameter of interest.
Prediction markets have been shown to be quite successful in predicting several outcomes (cf.
Wolfers and Zitzewitz, 2004). At the same time, it has been pointed out that successful market-
based predictions require certain preconditions to be met, including a sound market architecture
guaranteeing the heterogeneity of participants (Surowiecki 2004). Moreover, some critics object
that electoral markets simply mirror information available from other sources, i.e. election polls,
and add no new information (Erikson and Wlezien, 2008a).
Survey Models
We refer to the second kind of forecast models as “survey models” because their approach is
typical of election surveys that are sometimes used to predict election outcomes. In this model,
an appropriate random sample from the people who might affect future outcomes is required.
Then, the people included in this random sample are questioned about the ways in which they
intend to act (for example, vote in an election or purchase consumer goods). Then, the
distribution of behavioral intentions is interpreted as a forecast of the future outcome.
This procedure assumes that the sample is not biased and respondents’ future behavior does not
differ systematically from their stated intention (e.g., Perry, 1979; Rattinger and Ohr, 1993). The
usefulness of this model thus critically hinges upon the quality of the sample, the right questions
to be asked in the survey, and the interval between the interview and the future outcome. Quite
obviously, undecided respondents are a source of potential obstacles in the analysis.
Statistical Models
5
The third method builds on statistical models of the outcome of interest
2
. Using some kind of
time-series analysis, univariate models aim at detecting past regularities in the outcome variable
are then used to predict its future development. Multivariate models capture the relationship
between the outcome variable and several predictor variables. Whereas data-driven models
simply aim at detecting empirical relationships, theory-driven models identify predictors that can
be linked to the outcome in theoretically meaningful ways. In this vein, the vote share of an
incumbent party might be modeled as a function of the state of the economy several months
before an election, the results of trial heat polls, and the length of incumbency (for a variety of
electoral forecast models see, e.g., Campbell 2008; Erikson and Wlezien, 2008; Hibbs, 2008;
Holbrook, 2008). Having established a robust empirical model, predicting a future outcome
requires filling in relevant information on predictor variables and then calculating the dependent
variable (for a discussion of statistical issues in predictions see, e.g., Brandt et al., 2011;
Montgomery et al., 2012).
The success of statistical predictive models hinges upon the robustness of the empirical
relationships, in particular, the patterns detected in the past are assumed to hold in the future. In
the absence of a structural break, such predictions are likely to prove valuable. Yet, the absence
of structural breaks cannot be taken for granted. For example, an external shock might alter the
relationship between gross income and gross demand, or a new party might change the logic of
party competition and vote choice. Put differently, the success of statistical models crucially
hinges upon the assumption that the future closely resembles the past.
Forecasting Models with Social Media
In principle, the identified types of forecasting models can be adapted to and/or used in the
context of social media. Yet, it remains unclear what type of model best suits the characteristics
and the fabric of social media data.
Social media allow users to interact, to share content, and to create content collectively (O’Reilly,
2005; Shirky, 2008; Gauntlett et al., 2011). Social media comprise, inter alia, weblogs, social
networking sites, and platforms for music, video, and photo sharing. Every move users make on
social media is documented on machine readable formats. When analyzing these data, their
origin must be taken into account. In particular, Internet users and, even more so, users of social
media have voluntarily decided to use these applications and thus differ from the population at
large in terms of demographic characteristics, socio-economic variables, and socio-political
attitudes (e.g., Hargittai and Hinnant, 2008).
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2
This approach, also labeled “econometric models”, is common place, for instance, in
economics (e.g., Clements and Hendry, 2011; Hendry and Ericsson, 2001; Granger et al., 2006)
or electoral forecasting (e.g., Lewis-Beck and Rice, 1992; Campbell and Garand, 2000).