Understanding the massive open online course (MOOC)
student experience: An examination of attitudes, motivations,
and barriers
Heather B. Shapiro
a
, Clara H. Lee
a
, Noelle E. Wyman Roth
b
,
1
, Kun Li
a
,
Mine Çetinkaya-Rundel
a
, Dorian A. Canelas
a
,
*
a
Duke University, Durham, NC 27708, United States
b
Stanford University, Stanford, CA 94305, United States
article info
Article history:
Received 22 April 2016
Received in revised form 28 January 2017
Accepted 2 March 2017
Available online 6 March 2017
Keywords:
Adult learning
Lifelong learning
Teaching/learning strategies
Post-secondary education
Media and education
Online learning
Qualitative analysis
Sentiment analysis
NVivo
abstract
During the widespread development of open access online course materials in the last two
decades, advances have been made in understanding the impact of instructional design on
quantitative outcomes. Much less is known about the experiences of learners that affect
their engagement with the course content. Through a case study employing text analysis of
interview transcripts, we revealed the authen tic voices of participants and gained a deeper
understanding of motivations for and barriers to course engagements experienced by
students participating in Massive Open Online Courses (MOOCs). We sought to understand
why learners take the courses, specifically Introduction to Chemistry or Data Analysis and
Statistical Inference, and to identify factors both insid e and outside of the course setting
that impacted engagement and learning. Thirty-six participants in the courses were
interviewed, and these students varied in age, experience with the subject matter, and
worldwide geographical location. Most of the interviewee statements were neutral in
attitude; sentiment analysis of the interview transcripts revealed that 80 percent of the
statements that were either extremely positive or negative were found to be positive
rather than negative, and this is important because an overall positive climate is known to
correlate with higher academic achievement in traditional education settings. When de-
mographic data was added to the sentiment analysis, students who have already earned
bachelor's degrees were found to be more positive about the courses than students with
either more or less formal education, and this was a highly statistically significant result. In
general, students from America were more critical than students from Africa and Asia, and
the sentiments of female participants' comments were generally less positive than those of
male participants. An examination of student statements related to motivations revealed
that knowledge, work, convenience, and personal interest were the most frequently coded
nodes (more generally referred to as “codes”). On the other hand, lack of time was the most
prevalently coded barrier for students. Other barriers and challenges cited by the inter-
viewed learners included previous bad classroom experiences with the subject matter,
inadequate background, and lack of resources such as money, infrastructure, and internet
access. These results are enriched by illustrative quotes from interview transcripts and
* Corresponding author.
E-mail addresses: heatherbshapiro@gmail.com (H.B. Shapiro), claralee92@gmail.com (C.H. Lee), noellewymanroth@gmail.com (N.E. Wyman Roth),
kkunlli@gmail.com (K. Li), mine@stat.duke.edu (M. Çetinkaya-Rundel), dorian.canelas@duke.edu (D.A. Canelas).
1
Current address: QSR International, 55 Cambridge Street, Burlington, MA 01803, United States.
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu
http://dx.doi.org/10.1016/j.compedu.2017.03.003
0360-1315/© 2017 Elsevier Ltd. All rights reserved.
Computers & Education 110 (2017) 35e50
compared and contrasted with previous findings reported in the literature, and thus this
study enhances the field by providing the voices of the learners.
© 2017 Elsevier Ltd. All rights reserved.
1. Introduction
Massive Open Online Courses (MOOCs) are large-scale, open-access classes taught by university faculty via the internet
using a variety of techniques such as weekly lecture videos/webcasts, online assessments, discussion forums, and even live
video chat discussions and help sessions. Since the coinage of the term MOOC in 2008, this new avenue for education has
raised a great deal of excitement and controversy in academia; these discussions have been recently chronicled and reviewed
(Ebben & Murphy, 2014; Rhoads, Camacho, Toven-Lindsey, & Lozano, 2015). Some assert that these online courses carry the
potential to revolutionize the boundaries of modern learning by making high-quality education available to a much more vast
pool of students (Waldrop, 2013). To this end, early instructors using this format enthusiastically proclaimed “ …there's no
reason to limit the geographical boundaries. Anywhere there is an Internet connection, students can log-on to learn and get
help” (Moore & Janowicz, 2009, p. 4). MOOCs can also serve as an outlet for worldwide university outreach, expanding av-
enues for providing free, credible information to the general public. For example, some suggest that MOOCs will improve
science and health literacy awareness and discussion among the public (Goldberg et al., 2015; Leontyev & Baranov, 2013). The
power of a MOOC can be realized “especially on taboo subjects such as acquired immunodeficiency syndrome (AIDS),
tuberculosis, and contraception” (Liyanagunawardena & Williams, 2014, p. 11). In cases where rapidly emerging fields are led
by a small group of specialized experts, such as pharmacogenomics, MOOCs offer a route to efficiently improve in depth
education for a larger group of students than can be handled by individual faculty at all institutions (Ma, Lee, & Kuo, 2013). In
professions where continuing education is desirable or required, MOOCs also provide an efficient venue for adult working
professionals to gain new skills or stay current with new developments in their field. To this end, several MOOCs has been
certified for the purpose of providing continuing education to adults in professional fields such as K-12 teaching (Vivian,
Falkner, & Falkner, 2014), and physicians are beginning to promote MOOCs for continuing medical education, particularly
for medical practitioners in remote locations (Murphy & Monk, 2013). Clearly MOOCs have tremendous potential for the
promotion of life-long learning beyond the traditional classroom.
On the other hand, not all academics welcome MOOCs with open arms and view the possible future scenarios through
such rose-colored glasses. Some educators fear that a rapid proliferation of MOOCs could compromise the quality of learning
and lead to a deterioration of the post-secondary education system. These critics point to the importance of face-to-face
classroom engagement, laboratory, clinical, or fieldwork, and other aspects of the college experience outside of the pur-
view of formal coursework that would be difficult or impossible to replicate online (Cooper & Sahami, 2013; Harder, 2013;
Martin, 2012; McNutt, 2013). While most will now admit that online resources and online learning have a rapidly expand-
ing place in higher education, the current common platform configurations (Fidalgo-Blanco, Sein-Echaluce Lacleta, & García-
Pe
~
nalvo, 2015) and other limitations to synchronous, hands-on, and face-to-face experiences mean that MOOCs cannot so
easily replace higher education as we know it today. Others point to the relatively advanced education levels (Emanuel, 2013)
or high socioeconomic status (Hansen & Reich, 2015) of a large percentage of early participants to dampen the claim that
MOOCs are the solution to widening access to education. Moreover, since the overwhelming majority of MOOCs constructed
to date are in English, language access can be added to the technology access barrier for many populations
(Liyanagunawardena & Williams, 2014), although the landscape in this regard is rapidly evolving. Another set of concerns
cited revolves around the potential impact of MOOCs on academic life. Opponents of the MOOC movement point out that they
contribute to the casualization of academic labor and threaten current institutions of higher education (Kolowich, 2013;
Rhoads et al., 2015). Certainly the impact of MOOCs on the economic models of higher education constitutes a subject of
intense interest (Hollands & Tirthali, 2014; Hoxby, 2014), and some find the uncertainty in that arena unsettling. These and
other ethical implications of MOOCs, including cheating/plagiarism or research ethics involved for the large datasets pro-
duced by human subjects, have been recently explored (Marshall, 2014).
1.1. Prior research
Distance, online, and other forms of e-learning have a rich history of research, reviews of that research, and development
of models and frameworks (Arbaugh et al., 2009; Bernard et al., 2004; Childs, Blenkinsopp, Hall, & Walton, 2005; Gikandi,
Morrow, & Davis, 2011; Leacock & Nesbit, 2007; Lee & Choi, 2011; Roca, Chiu, & Martínez, 2006; U.S. Department of
Education, 2010; Zawacki-Richter, B
€
acker, & Vogt, 2009). However, MOOCs and research specifically related to MOOC ped-
agogies and learner outcomes are relatively recent developments in this arena. The MOOC research literature prior to 2012
has been reviewed elsewhere (Liyanagunawardena, Adams, & Williams, 2013).
Researchers from multiple disciplines have studied students' demographics, their performances, MOOC retention rates,
best practices for course design and pedagogy, etc. Most studies about MOOCs to date have employed primarily quantitative
or mixed-methods, such as analysis of course statistics and student survey data. For example, some of the early published
H.B. Shapiro et al. / Computers & Education 110 (2017) 35e5036
research examined student learning in edX's first MOOC by evaluating course component access and completion, time spent
on each resource online, scores on assignments, persistence, and some student demographic and survey data such as location,
age, and selected reasons for enrolling (Breslow et al., 2013; DeBoer, Stump, Seaton, & Breslow, 2013). More recent studies
have used survey data to explore education research topics such as students' self-regulated learning behaviors in the context
of MOOCs (Hood, Littlejohn, & Milligan, 2015). Research into innovative course designs is beginning to show promise in
increasing course completion rates (Fidalgo-Blanco et al., 2015; Fidalgo-Blanco, Sein-Echaluce, & García-Pe
~
nalvo, 2016).
Researchers examining the effects of MOOCs on participants who cannot afford formal post-secondary education found
that these learners were much less likely to have a college degree and were much more likely than a comparison group to
enroll in a MOOC due to reasons of geographic isolation (Dillahunt, Wang, & Teasley, 2014). Students who self-reported that
they could not afford a formal college education were more likely than the comparison group to be using a MOOC to see if they
wanted to enroll in a more formal college course and were also more likely to be awarded a certificate of achievement
(Dillahunt et al., 2014).
Researchers examining the pedagogies employed in a cross-section of MOOCs determined that an objectivist-individual
approach (Arbaugh & Benbunan-Fich, 2006) was the most common framework and was used in all 24 MOOCs examined
(Toven-Lindsey, Rhoads, & Lozano, 2015). They conclude that MOOC creators should “ …strive for more creative and
empowering forms of open online learning” (Toven-Lindsey et al., 2015, p. 1).
Qualitative work in MOOC research is just beginning. The affective domain has been investigated through qualitative
methods examining student writing in assignments and on discussion forums in MOOCs (Comer, Clark, & Canelas, 2014).
Motivation to learn in MOOCs has recently been studied from various perspectives, such as by examining aspects of language
and social engagement (Barak, Watted, & Haick, 2016). Finally, novel research methods such as blog mining have been
developed to gauge the tenor of recent online discussions of MOOCs in various nontraditional publication outlets (Chen, 2014).
Motivation plays a key role in persistence and the extent of learning in all education environments, and there is a rich body
of literature on the complex relationships between students' motivations, attitudes, and levels of engagement in a variety of
learning contexts. Motivation theory is often invoked to explain why individuals choose to participate in certain tasks and
their related effort level (Bandura, 1989a, b; Graham & Weiner, 1996; Keller, 1979). Participation in MOOCs currently falls
heavily into the category of “voluntary learning,” so therefore it follows that motivation is especially important in determining
individual differences in both total time spent on learning and effort intensity (Lei, 2010). While reviewing the rich history of
the development of motivation theory in various contexts is beyond the scope of this contribution, interested readers are
referred to reviews of motivation studies in online learning, in general (Bekele, 2010; Hart, 2012), and a recent detailed
discussion of motivation research and its application to MOOCs, in particular (Barak et al., 2016; Ferguson & Clow, 2015;
Kizilcec & Schneider, 2015 ).
1.2. The present study
Herein, we probe the future of online education through mixed-methods research. This involved analyzing surveys and
interview transcript data from semi-structured interviews of participants in two Coursera courses, Introduction to Chemistry
and Data Analysis and Statistical Inference, to gain a deeper understanding of the MOOC student experience. Our initial
research questions were very broad:
1. Who are the students taking MOOCs?
2. What motivates students to take MOOCs?
3. What can we learn from students that may help improve the MOOC experience?
We aimed to use semi-structured interviews and qualitative coding techniques to probe more deeply into the relationship
between motivation and engagement and explore emerging themes that to date have been only examined in a cursory way
through surveys. For example, what challenges do participants face in the effort to have full engagement with and deep
learning of the course materials? By understanding the motivations for students to take MOOCs, as well as the challenges one
might face throughout such courses, we aim to provide insight into a poorly studied topic, providing a foundation for further
research. Ultimately, such research could lead to platforms and pedagogies that maximize the number of positive student
experiences and provide a better learning environment for those who choose to enroll.
This project examines students' motivations and perceived barriers and challenges by interviewing students in the two
courses. The project can contribute to the literature on MOOC students' motivation and provide deeper understandings of
how MOOC designers can help learners overcome their perceived barriers in taking the course.
1.2.1. Description of courses
The Introduction to Chemistry course is taught by the sixth listed author of one of the submitting institutions. The session-
based course seeks to reach students with little to no background in the subject in order to prepare the students for further
study in chemistry, which is needed for many science, health, and policy professions. Students are introduced to chemical
problem solving involving topics such as atoms, molecules, ions, the periodic table, stoichiometry, chemical reactions,
bonding, thermochemistry, and gas laws. The course features videos, discussion forums, problem sets, quizzes, and exams as
H.B. Shapiro et al. / Computers & Education 110 (2017) 35e50 37
well as an optional writing project assessed by peers. It was adapted from a campus-based course described in detail else-
where (Canelas, 2015; Hall, Curtin-Soydan, & Canelas, 2014). Since the completion of the data acquisition for this study, the
course has been split into two shorter courses: Introduction to Chemistry: Reactions and Ratios and Introduction to Chemistry:
Structures and Solutions.
The fifth listed author of one of the submitting institutions teaches the Data Analysis and Statistical Inference course. The
session-based course aims to introduce students with little to no experience to the discipline of statistics while learning how
to collect data, how to analyze data, and how to use data to make inferences and conclusions about real world phenomena.
The course features videos, quizzes, and exams as well as data analysis labs in the R statistical language and an independent
data analysis project assessed by peers. Since the completion of the data collection for this study, this course has also been
split into a series of courses that make up the “Statistics with R” Coursera specialization.
2. Theoretical framework
This work is comprised of mixed-methods research guided by the literature and scholarship in motivation theory, e-
learning, and distance education theory (Bernard et al., 20 04; Sun, Tsai, Finger, Chen, & Yeh, 2008). We draw inspiration from
the e-learning theoretical framework outlined by Aparicio and coworkers, who use an overlapping domains conceptual
framework consisting of people (stakeholders), e-learning technologies including those designed for content and commu-
nication, and e-learning activities including pedagogical models and instructional strategies (Aparicio, Bacao, & Oliveira,
2016). This is a comparative case study using the framework method for thematic development and analysis of semi-
structured interview data.
The theoretical framework for the study described herein involves overlapping domains of student motivations, barriers/
challenges, and demographics. Motivations for students to engage in a course can vary greatly from extrinsic to intrinsic in
online learning environments (Hartnett, St. George, & Dron, 2011). According to the pre-course surveys that the university
sent out to their MOOC students, the majority of students signed up for a particular MOOC because they considered it fun or
they were interested in the topic (Belanger & Thornton, 2013). Other reported motivations for taking a MOOC included (1)
supporting lifelong learning or gaining an understanding of the subject matter with no particular expectations for completion
or achievement, (2) convenience, often in conjunction with barriers to traditional education options, and (3) to experience or
explore online education (Belanger & Thornton, 2013). Another study using survey data reveals that over half of students in an
edX course chose to enroll in that course because they wanted to learn the knowledge and skills offered by that course
(DeBoer et al., 2013). MOOC students' varied motivations also affect their participation in course activities. Yang (2014) found
a positive correlation between MOOC students' intrinsic motivation and their participation in online discussions, but this
relationship only occurred during later phase in the MOOC. Similarly, Halasek et al. (2014) stated “participants' interests and
personal motivation determined whether and how they engaged with course materials…” (p. 162).
While online education provides learners greater opportunities to access learning resources, it also generates new chal-
lenges to online learners (Anderson, 2008; Song & Hill, 2007). Lack of prompt responses from instructors, too many channels to
obtain information online, lack of effective self-regulated learning skills, procrastination, and superficial participation in online
discussion are some of the common challenges for online learners (Song & Hill, 2007). MOOCs usually have thousands or tens of
thousands of learners, so those challenges listed above can be even more obvious for MOOC learners. For example, because of
the instructor to student ratio, many MOOCs use the discussion forum for students to ask questions instead of direct emails toor
conversations with the instructor. Instead, peers are answering questions more often than the instructor. Additionally, many
MOOC learners drop courses because of low self-regulated learning skills and procrastination (Diver & Martinez, 2015).
Using our theoretical framework, we will probe motivations, barriers, and demographics as well as the overlap of these
domains to draw conclusions about the experience of participants in the MOOCs.
3. Data and methods
3.1. Data collection and description of sample
Participants were recruited from both courses through announcements on the Coursera course sites, email messages, and
posts on the courses' Facebook pages (Appendix A). The first stage of the interview subject recruitment was a survey in which
students were asked about their spoken English fluency and whether or not they were willing to participate in an interview
(Appendix B, document 1). At the time the survey was sent out, the Statistics course had roughly 10,000 students participating
on a consistent basis in the course and the response rate was roughly 15%. The response for the Chemistry course was slightly
higher, with a 28% response rate from the roughly 3000 active students at the time the survey was sent out.
We used stratified sampling to identify a diverse set of interviewees based upon gender, age, educational background,
income, and geographic location. We interviewed 20 students from Chemistry and 16 from Statistics; Table 1 shows a
breakdown of the participants' demographics for each class. The interviewer guided the participant through a series of
questions according to the interview guide (Appendix B, Document 2) over an audio-only internet line (via Skype) with
interviews lasting from 30 to 45 min. Using the semi-structured interview format, questions from the guide were supple-
mented by additional follow up questions as they arose. For instance, interviewees were asked about their educational
background prior to the course, how they planned to use the course material after completion of the course, and how
H.B. Shapiro et al. / Computers & Education 110 (2017) 35e5038
important the course was to them. The interviews were recorded, transcribed via a professional transcription service (Cas-
tingWords), and then analyzed with the aid of statistical and qualitative analysis software: RStudio and NVivo, respectively. To
protect participant privacy, all data was de-identified to remove personal descriptors (names, emails) prior to analysis.
All subject recruitment and consent processes (Appendix C) followed protocol C0103, “Understanding the MOOC Student
Experience through Qualitative Research Interviews,” which was approved prior to beginning subject recruitment and data
collection by one of the submitting universities' Institutional Review Boards (IRB).
We did not analyze the data for completion rates of the interview participants. As noted by DeBoer and colleagues, the
extremely low barrier for “enrollment” in a MOOC means that traditional academic terms such as enrollment and partici-
pation need to be reconceptualized in order for them to regain meaning in this context (DeBoer, Ho, Stump, & Breslow, 2014).
This has caused researchers to begin analyzing retention data in alternative ways, such as by considering students who
registered for verified certificates as a separate track (Engle, Mankoff, & Carbrey, 2015) or by using latent profile analysis (a
form of cluster analysis with a time dimension) (Wiebe, Thompson, & Behrend, 2015).
3.2. Sentiment analysis methods
In order to understand the overall attitude of the interviewee responses, we analyzed the transcript text files using an
automated sentiment analysis method. The analysis cross-referenced the words in each of the transcripts with an opinion
lexicon of both positive and negative words (Hu & Liu, 2004). The sentiment analysis followed the approach suggested by
Breen (2014). Using this method, we calculated the sentiment score for each sentence in the transcript of interviewee
comments using the following formula:
Score ¼ Number of positive words Number of negative words
If Score > 0, the sentence is considered to have an overall ‘positive’ opinion
If Score < 0, the sentence is considered to have an overall ‘negative’ opinion
If Score ¼ 0, the sentence is considered to have an overall ‘neutral’ opinion
A breakdown and quantitative analysis of sentiment scores by individual properties was performed to understand the
overall attitude of different groupings of people who were interviewed.
3.3. Qualitative analysis methods
We also analyzed all interview transcripts through a manual text coding process using a framework method with
emerging thematic analysis via NVivo, a qualitative data analysis software program. Prior to coding, interviewee names were
Table 1
Description of Sample: Interview participants.
Chemistry
n (%)
Statistics
n (%)
a
Total
n (%)
a
Gender Female 9 (45%) 4 (25%) 13 (36%)
Male 11 (55%) 12 (75%) 23 (64%)
Highest Level of Education Less than HS 1 (5%) 0 (0%) 1 (3%)
Some College 3 (15%) 0 (0%) 3 (8%)
Associate's Degree 1 (5%) 0 (0%) 1 (3%)
Bachelor's Degree 8 (40%) 11 (69%) 19 (53%)
Graduate Degree 7 (35%) 5 (31%) 12 (33%)
Age 18e24 6 (30%) 2 (13%) 8 (22%)
25e34 6 (30%) 10 (63%) 16 (44%)
35e44 3 (15%) 2 (13%) 5 (14%)
45e54 2 (10%) 1 (6%) 3 (8%)
55þ 3 (15%) 1 (6%) 4 (11%)
Annual Income <$20,000 6 (30%) 5 (31%) 11 (31%)
20,000e34,999 4 (20%) 2 (13%) 6 (17%)
35,000e49,999 3 (15%) 0 (0%) 3 (8%)
50,000e74,999 1 (5%) 6 (38%) 7 (19%)
75,000e99,999 2 (10%) 1 (6%) 3 (8%)
100,000þ 4 (20%) 2 (13%) 6 (17%)
Continent Africa 1 (5%) 2 (13%) 3 (8%)
Americas 13 (65%) 3 (19%) 16 (44%)
Asia 3 (15%) 6 (38%) 9 (25%)
Europe 3 (15%) 5 (31%) 8 (22%)
a
Percentages provided do not always add up to 100% due to rounding.
H.B. Shapiro et al. / Computers & Education 110 (2017) 35e50 39
