An Interactive E-book with an Educational Game
for Children with Developmental Disorders: A Pilot
User Study
Nirvana Pistoljevic
Teachers College
Columbia University
New York NY 10027, USA
Email: np2127@tc.columbia.edu
Vedad Hulusic
LTCI, T
´
el
´
ecom ParisTech
Universit
´
e Paris-Saclay
Paris, France
Email: vedad.hulusic@telecom-paristech.fr
Abstract—Children diagnosed with Autism spectrum disorder
(ASD), as one of the most complex neurodevelopmental dis-
abilities, are characterized by different brain and functioning
development, distinct interaction with the environment and
different learning patterns, language and social skills impair-
ments, and repetitive auto-stimulating restricting behaviors. It
has been shown that computer-assisted intervention is much more
attention captivating and interesting to children compared with
a classic approach to teaching, allowing for faster acquisition of
skills. This makes these tools and the technology highly suitable
for teaching children with autism basic developmental skills. In
addition, interactive electronic books showed positive outcomes
for comprehension and information acquisition in children with
ASD, while decreasing inappropriate children behavior in the
classroom. In this paper a pilot user study on an e-book with
an embedded educational game for children with developmental
disorders was presented. The results show that the e-book
can be efficiently used for teaching children with ASD basic
developmental skills and that the learned skills can be efficiently
transfered to new media and environments. The framework will
provide preschool children with and without disabilities with
appropriate educational software, to build up their early cognitive
abilities and school readiness skills, and promote incorporating
technology as part of the educational and pedagogical process in
schools.
I. INTRODUCTION
Children diagnosed with developmental disabilities account
for a large spectrum of (neuro)developmental disorders like
autism, ADHD, intellectual disabilities, etc. The main char-
acteristic of this neuro-atypical development is that these
children and later adults, interact with the environment in
different ways, their brains develop and function differently,
so they learn in a different way. Autism spectrum disorder
(ASD) is one of the most complex neurodevelopmental disor-
ders, which characteristics like impaired language and social
skills development, and repetitive auto-stimulating restricting
behaviors, interfere with ones overall functioning [1]. The
spectrum means that overall functioning of the individuals
with ASD can be anywhere between high functioning or mild
severity, when only some language delay or lack of social
skills is apparent, to severe disability with no vocal verbal
communication skills or eye contact and interest in social
interaction. This, in turn, greatly affects the way individuals
on the spectrum learn and the way learning environment and
curricula need to be designed and arranged.
Computer science contribution to this population has been
great already, but there is still a vast land of research oppor-
tunities barely tapped into. Applications of technologies like
eye-tracking [2], [3] and skin conductivity sensors [4], [5] have
just begun to shed the light on the way people with autism,
for example, visually scan their environment and learn, and
what levels of anxiety they feel during those interactions.
Numerous studies have demonstrated that children with
ASD acquire skills faster when instruction is delivered via
computer vs. the traditional teaching strategies, and are more
interested and motivated to learn through computer-assisted
instruction [6], [7]. In recent years, significant amount of
work in serious games community has contributed in this
domain [8].
In this work a pilot user study on teaching effectiveness
of the previously developed interactive electronic book for
children with and without autism [35] is presented. The study
was based only on the game portion of the interactive e-book.
We wanted to test whether children with disabilities like ASD
will acquire basic learning concepts (i.e. pointing to colors
and numbers or basic vocabulary from the story) by going
through the e-book and playing the game and will they be
able to transfer that knowledge, i.e. is it generalizable.
There are several contributions of this work. The results
of the pilot study indicate that this highly motivating and
engaging game-based environment can be used for teaching
children with ASD basic developmental skills. Furthermore,
it has been shown that the learned skills can be efficiently
transfered to new media and environments. Finally, this work
promotes incorporating technology as part of the educational
and pedagogical process in all schools.
II. RELATED WORK
It has been shown that computer-assisted interventions can
provides children with developmental disorders an opportunity
to work on building of basic developmental skills. In addition,
it allows them to practice these on their own time, with or
without supervision, and to adjust the appropriate pace while
at the same time receiving feedback in form of reinforcement
and correction. Bosseler and Massaro developed and tested
via two separate experiments a computer-animated program
for teaching children with ASD vocabulary and grammar.
They found that children with ASD learned novel vocabulary
with a computer-animated agent and they noted the active
participation of children in the process of learning, transfer and
use of the novel language in a natural, untrained environment,
which is the key of learning any skill [9].
Kebritchi and Hirumi in a comprehensive educational games
review pointed out that this type of games are designed
taking into account several learning theories such as learning
by doing, discovery learning or constructivist learning [10].
Skinner [11], looking from a point of view of the science of
learning, pointed out how in video games players behavior is
guaranteed to be reinforced, because the player contact salient
and immediate consequences, and are almost guaranteed to
be successful. The importance of the immediate feedback a
teaching machine can provide vs. teacher teaching a large
group of students and never being able to do so, was em-
phasized by Pressy [12] in the 20’s as a main contribution
of computerized instruction to the education. In addition,
enabling students to advance through curricula at their own
pace, permitting them to play an active role in learning vs.
passive listener role is shown to be highly motivating for a
learner [12], [11]. The feedback, being a correction and leading
or prompting a player to a correct response or being a positive
reinforcement to strengthen the responses in the future, seems
to be a key to learning [13], [11], [14], [15]. The consequence,
a component of the three-term contingency or a learn unit
is what effects the behavior in the future. The appropriate
and immediate consequence based upon student behavior,
when properly delivered will increase the likelihood of future
correct responding to the same or similar stimuli. Therefore,
the intact learn units (LUs), complete antecedent-behavior-
consequences, should be the aim of all instruction, computer
assisted or not. The interlocking three-term-contingency has
been a proven measure of a teacher (i.e. machine)-student
interaction upon which all learning is based on [16], [14],
[15].
The notion of serious games and its definition are not new
[17]. Generally, they are defined as games designed for pur-
poses other than leisure and entertainment, such as education,
health, training, simulation, cultural heritage and other. There
are many examples of serious games that have been proven
to be efficient in achieving their goals while providing a fun,
engaging and informal, game-like environment. Game-based
learning and educational serious games combine gaming and
learning, and are proven to be effective way of computer-aided
learning [18][19]. A study by Kebritchi et al. showed that
modern mathematics computer games have positive effects
on mathematics achievement and class motivation [20]. In
another example, the Re-Mission serious game allows young
people with cancer to learn about it and its treatment [21].
A systematic and extensive review of serious games, with an
emphasis on the positive aspects of gaming on learning and
skill enhancement, has been presented in work by Connolly
et al. [22]. In their paper they showed that playing computer
(serious) games can have multiple positive cognitive, behav-
ioral and motivational effects. Another study by Annetta et
al. showed that such games can enhance engagement and
motivation in children with learning difficulties or attention
disorders [23].
Another, closely related, subcategory of serious games are
those mainly used for health treatment. These serious games
are not used only for treatment of diseases, but also for
physical, mental, and social well-being treatments. These
include obesity [24], [25], phobias [26], [27], diabetes [28],
[29], autism [30], [31], [32] and many others. A complete
literature survey on serious games for health can be found
in work by Wattanasoontorn et al. [33], that also classifies
them by game purpose, functionality, stage of disease and
by player’s wellness and analyzes each game based on dif-
ferent functionalities. Serious games for ASD treatment are
mainly aimed at teaching basic developmental skills, including
communication, social behavior and conversation, imaginative
skills, sensory integration, emotional expressions, etc. [30],
[31], [32], [34], [35]. An extensive overview of serious games
for treatment of ASD, along with a classification according to
technology platform, computer graphics (2D vs 3D), gaming
aspect and interaction type, is presented by Zakari et al. [8].
One of the Electronic picture storybooks often include
motion pictures, sounds, and background music instead of
static pictures, and hotspots that label/define words when
clicked on. Smeets and Bus examined whether these animated
e-books aid word learning and story comprehension by com-
paring outcomes for kindergarteners across four experimental
groups: static e-books, animated e-books, interactive animated
e-books, and a control group that just played non-literacy
computer games [36]. They found that children acquired
most vocabulary after reading interactive animated e-books,
followed by non-interactive animated e-books and then static
e-books. In a similar study [37], but with children with
ASD, reading interactive e-books (on iPads) vs. printed books
was compared base on the comprehension and information
acquisition. They found positive outcomes for each participant
in a group using interactive e-books and teachers survey
reveled that the students found iPads motivating, asked to use
them as reinforcers, which in turn decreased their inappropriate
behaviors in the classroom.
III. E-BOOK DEVELOPMENT
The main objective of this project was to develop an
educational interactive e-book for early childhood stimulation
and to evaluate its effectiveness on learning numbers, colors,
novel vocabulary, identification, counting and responding to
inference questions. The e-book introduces children with and
without ASD a fun, enjoyable, interactive and educational
game-based environment. The e-book has been developed for
the web using the following technologies: HTML5, CSS3,
JavaScript, jQuery and AJAX, which makes it suitable for
practically any device (desktop/laptop computer, tablet, smart
phone) that has internet access.
A. Theoretical background
In order to create serious games that foster learning there
are some core principles of design in addition to understanding
and applying principles of the learning theory. In the design
of the interactive e-book with an embedded educational game,
there are several important principles: immersive storylines,
goals based on targeted skills to be taught, feedback about
progress, progressing through levels of difficulty, individual-
ization, and provision of choice [38], [39], [40]. These studies
showed that learning opportunities need to be integrated with
the story content, a narrative needs to be built to support
learning goals and encourage players to develop emotional
connection with the characters, so that they will be motivated
for learning. In developing the e-book, we paid close attention
to these guidelines, and worked with the children story writer
to create a main character, a wolf who falls in love and
becomes a boy, while learning goals of recognizing colors,
numbers, letters, counting, vocabulary building, etc. were
embedded within the story, see Figure 1. Throughout the story
narrated by children in two languages, players can choose the
language of narration and game questions, whether to hear
a narrator or to rely only on the text, and whether to start
the game portion on the page or to only explore embedded
interactive functions within the story. Yet another level of
individualization is achieved by three levels of difficulty all
game instructions/questions have. Some are selection respond-
ing, just pointing to the color or the picture, while others are
abstract and inference questions like interpreting the feelings
of the character in the scene, see Table II.
TABLE I
THREE GROUPS OF QUESTIONS IN THE E-BOOK.
Question
group
Question type / Educational goal
1
Selection-based questions with varied antecedents (i.e.
“point to”, “find”, “where is”, “touch”)
2
1:1 correspondence questions for teaching children to
count objects
3
Production responding, inference questions, where the
child reads a short text and makes conclusions based
on the story comprehension
B. Web-based implementation
The interactive e-book was developed using web technolo-
gies, allowing an easy-to-use, real-time interaction through the
web-based front end. The game contains 13 screens (a cover
plus 12 playable), corresponding to the same number of pages
in the printed book. All scenes are displayed through a wrapper
index.html page, into which the requested corresponding page
is loaded. The index.html page contains all the logic for the
questions and answers, the language selection and for scene
traversal, while the individual scenes’ html files contain event
listeners which are responsible for the page interaction, such
as playing the sounds, hiding/showing the page elements,
selecting the questions, and initiating the corrections. All
the styling, including visual appearance, sizes, positions and
effects, is done using Less elements preprocessor.
The whole game is implemented bilingually, in
Bosnian/Serbian/Croatian (native) and English, so that
the language can be toggled at any point withing the game,
allowing for learning a foreign language. Each scene contains
three question, two of which have three predefined answers
displayed in a random order. Other questions, e.g. “where
is” or “point to”, require interaction with the active scene
elements, see Figure 1. For each correct answer the applause
sound is played and one of the text responses (“Great job!”,
“Excellent!”, “That’s correct!”) is randomly provided. In
case of the incorrect answer the correction is invoked. More
details on the game implementation can be found in [35].
IV. A PILOT USER STUDY
The creation and the interdisciplinary design of the inter-
active e-book was described in detail in the paper published
by Hulusic and Pistoljevic [35]. It was designed to attract
and keep attention of a learner, to engage him/her by features
additional to the text that a player can choose to be read in
two languages. Additional features, for example, were auditory
stimulation when clicked on a certain object in the scene:
naming the object or making a related sound, or a multi layered
questions about the scene or story read for comprehension
and vocabulary building. There were three questions that went
along with the text of the story on each page, increased
in difficulty to accommodate different levels of learners (a
preschooler or a child with a disability, pre-reader vs. elemen-
tary school age typically developing child, a reader). With the
embedded game, our educational goals were to teach children
basic vocabulary, foreign language vocabulary and school
readiness concepts like identifying target objects (i.e. learn
the vocabulary), counting objects from 1 to 10, identifying
numbers 1-5, colors, and responding to inference questions
(i.e. comprehension of the text). First question on each page
was the easiest level question, a selection-based question with
varied antecedents (i.e. point to, find, where is, touch). Second
question was related to teaching children to count, using the
scene from the book and objects in it. The third level was the
production responding, an inference or an abstract thinking
question like whether someone was sad based upon the text
on that page the child read/heard being read. These questions
were based on the comprehension of the story, so for older
children it would require reading skills. The overall goal
of the interactive e-book was enriching the reading/listening
to narration, experience for children functioning on different
levels and to provide an engaging learning environment to
children with disabilities, especially multisensory needs of
children with ASD. This pilot user study focused only on the
36 questions embedded in the e-book. Following the reading
of the page, student was able to select to engage in the
game portion on the bottom of the screen (Figure 1) and to
respond to 3 questions per page base on the page visual and
Fig. 1. Three scenes from the e-book with the narrated text displayed and different question groups/layers: counting (left), pointing to (middle) and inference
question (right).
TABLE II
THE INSTRUCTIONS USED IN THE QUESTIONS, SORTED PER E-BOOK PAGE.
THE ACTUAL QUESTIONS WERE DIFFERENTLY FORMULATED TO REFLECT
THESE INSTRUCTIONS, E.G. THE FIRST QUESTION ON THE FIRST PAGE
WAS: “HOW MANY BIRDS CAN YOU FIND?”, WITH FIVE BIRDS ON THE
PAGE AND THREE PROPOSED ANSWERS: “4”, “5” AND “6” IN A RANDOM
ORDER. PT STANDS FOR “POINT TO” / “WHERE IS”.
Pg
no.
Question 1 Question 2 Question 3
1 Count 5 objects PT bird PT color red
2 PT wolf Count 6 objects PT Goldilocks
3 PT to number 4 PT millipede Boy/Girl Y/N question
4 PT water bucket Count 7 objects Select Wolf (inference)
5 PT letter C ID action waving PT color red
6 PT bush Count 8 objects Si the wolf sad Y/N
7 Count 9 objects PT mirror Does the fairy flies Y/N
8 Count 10 objects PT keyboard PT wolf
9 Count 4 objects PT popcorn Wolf has fullmouth Y/N
10 PT blue car PT glass Count 5 objects
11 PT yellow PT wolf PT glasses
12 Count 1 object PT heart
Is a boy wearing glasses
Y/N
textual input. Following the mastery of the game, we assessed
the transfer of the vocabulary and concepts learned through
playing the game by asking children to use the vocabulary
and concepts in their natural environments.
A. Participants
In this pilot user study, we tested only the game component
of the interactive e-book. Ten students, one girl and nine boys
participated in the pilot study, ranging in age from 4 to 7
years old. All students were diagnosed with autism and/or
other developmental delays with autistic elements. Children
attended a specialized evidence-based model of instruction
“EDUS-Education for All”, provided through public preschool
program for children with developmental disorders. Children
attended an evidence-based program that applied behavior
analysis to all aspects of teaching, for 5h daily and all had
individualized education plans and programs. Seven children
attended a classroom with a student to teacher to teaching
assistant ratio of 10:1:1, while three attended an 8:1:3 ratio
classroom. All students in EDUS programs were grouped
according to their levels of verbal behavior, so it would be
TABLE III
GENERALIZATION PROBE: THE QUESTIONS / TASKS, PERFORMED IN
PICTURES AND/OR BOOKS AND IN THE CHILDREN’S NATURAL
ENVIRONMENT. PT STANDS FOR “POINT TO”.
No. Question / Task No. Question / Task
1 Count 10 objects/manipulatives 14 PT “B”
2 PT RED 15 PT “C”
3 PT BLUE 16 PT boy
4 PT YELLOW 17 PT girl
5 PT “1” 18 PT bucket
6 PT “2” 19 PT Bush
7 PT “3” 20 PT mirror
8 PT “4” 21 PT Computer keyboard
9 PT “5” 22 PT popcorn
10 PT bird 23 PT glass
11 PT wolf 24 PT glasses
12. PT millipede 25 PT heart
13 PT “A”
safe to say that seven participants were speakers and listener,
emergent readers and writer, while three were on a lower
level of verbal behavior and were emergent speakers and
listeners. All instruction across all EDUS classrooms was done
via learn units, teacher delivering instruction, reinforcement
and corrections while continually taking data on all students
responses and behaviors. Same rules were applied during the
game playing while children sat at the computer table with the
teacher who had a data collection sheet and read the questions
from the game to the participants. Due to the age and level
of functioning, children were not able to read the questions
in the game to themselves. A teacher working with them
was instructed to only get their attention to the game, read
the questions and record student behaviors. He/she was not
allowed to help students in any way or to prompt or explain
any questions.
B. Design
The participants played the game with a teacher reading the
questions and taking data on the LUs (antecedent-behavior-
consequences). Only data on the game playing, not interacting
with the e-book, were recorded for the purpose of this study.
The instructions used in the data sheets during the data
collection for all 36 questions in the game is shown in Table II.
0
6
12
18
24
30
36
H.Z.
(97%)
N.H.
(94%)
A.S.
(92%)
A.K.
(97%)
E.H.
(97%)
M.A.
(94%)
E.S. (F)
(72%)
H.C.
(75%)
T.D.
(83%)
N.G.
(72%)
No. of correct achieved
Participant
Trial 1
Trial 2
Trial 3
Trial 4
Trial 5
Fig. 2. The number of correct answers achieved per participants across the trials. Bellow the participants’ initials the percentage of correct answers is
presented. Last for participants did not achieve the mastery. Participant E.S. was the only female (F) participant in the study.
Following the mastery, generalization probes were conducted.
They were conducted on non-screen media, that is across
different school environments and as responses to pictures in
other children books. Criterion for mastery of the game was
set at playing it once at a minimum of 90% accuracy, that is
33 out of 36 correct responses to the game tasks. The data
were looked at as total numbers of LUs (i.e. trials) required
for mastering the game, that is reaching the predetermined
criterion level. Numbers of LUs to criterion were calculated
as well, to tell us on average how many LUs children needed
to master the game. Comprehension and vocabulary were
only post-tested, to prevent that a repeated exposure to target
words in tests could explain the outcomes, to avoid practice
effect [41], [42], [36]. During the generalization probes, a total
of 25 vocabulary questions were tested. They were conducted
in children’s natural environment, during instruction and free
play, children were asked to point to or show target vocabulary
from the game from their environment or in hard cover books
and pictures in the classrooms. The questions/tasks from the
generalization probe data sheet are shown in Table III.
C. Interobserver agreement
Interobserver agreement or reliability observations were
conducted with an independent observer for a total of 25%
of all sessions of game playing, with a mean of 98,5%
(ranging from 92% to 100%). While the teacher was reading
the antecedents to the student and recording the students’
responses on a data sheet, an independent observer was doing
the same on a separate sheet without any communication with
the participant or the teacher. The interobserver agreement
scores were calculated by dividing the total numbers of paired
observer agreements by the total numbers of agreements plus
disagreements and then multiplying that number by 100 [43].
V. RESULTS
The results of the pilot user study with 10 par-
ticipants showed that the created software in native
(Bosnian/Serbian/Croatian) language was completely clear and
user-friendly for kids with special needs, and that it is sys-
tematically and developmentally appropriately sequenced for
learning, see Figure 2. Additionally, it was found that children
were able to generalize learned skills, through a transfer to new
media or environments, by using the vocabulary in other hard
cover books or by labeling 3D objects in their environment.
Their teacher also reported that children were very motivated
and enjoyed playing the game. For many of them, the game
became a reinforcer, i.e. they asked to (re)play it. After an
average of playing the game 4 times, all children showed a
significant increase in correct responding and reached levels
of over 72% accuracy, see Table IV. Six children mastered
the game, reaching the 90% or more correct responses. Due
to the objective limitations, such as time and access to the
children, the other 4 participants did not complete the game.
For 3 out of the 6 who completed, we conducted the post-
probes, maintenance probes to test for the transfer of learned
concepts to the natural environment. All students demonstrated
high transfer of vocabulary learned to other, non-computerized
settings.
Testing such games with children with ASD is highly time
and resource demanding, as the children is tested the game
only once per day and testing a game with one child has to
be performed by two teachers in an isolated environment and
in a suitable time of the day that complies with the children’s
daily teaching routines. Nonetheless, those that have not so
far, and other students not included in the study, will continue
playing it and will master it as part of their daily instruction.
VI. CONCLUSION AND FUTURE WORK
The e-book was designed to engage children with its
auditory-visual stimulation, related to the storyline presented
bilingually, with multi layered questions about the story. Our
educational goals were to teach children novel vocabulary,
counting, identifying numbers and colors, and responding to
inference questions.
According to National Center on Educational Statistics from
2001, approximately 90% of children ages five through seven
use computers in the USA [44]. Video games in particular
provide an interactive, self-paced and motivating tool that
presents concepts in a way that are engaging, informal, and
fun for children [45]. Education of children with disabilities is
