scispace - formally typeset

Journal ArticleDOI

Neurophysiological methods for monitoring brain activity in serious games and virtual environments: a review

01 Mar 2014-International Journal of Technology Enhanced Learning (Inderscience Publishers)-Vol. 6, Iss: 1, pp 78-103

TL;DR: The opportunities offered and challenges posed by neuroscientific methods when capturing user feedback and using the data to create greater user adaptivity in game are explored.
Abstract: The use of serious games and virtual environments for learning is increasing worldwide. These technologies have the potential to collect live data from users through game play and can be combined with neuroscientific methods such as EEG, fNIRS and fMRI. The several learning processes triggered by serious games are associated with specific patterns of activation that distributed in time and space over different neural networks. This paper explores the opportunities offered and challenges posed by neuroscientific methods when capturing user feedback and using the data to create greater user adaptivity in game. Existing neuroscientific studies examining cortical correlates of game-based learning do not form a common or homogenous field. In contrast, they often have disparate research questions and are represented through a broad range of study designs and game genres. In this paper, the range of studies and applications of neuroscientific methods in game-based learning are reviewed.

Summary (6 min read)

MURDOCH RESEARCH REPOSITORY

  • This is the author’s final version of the work, as accepted for publication following peer review but without the publisher’s layout or pagination.
  • Http://dx.doi.org/10.1504/IJTEL.2014.060022.

6 (1). pp. 78-103.

  • Http://researchrepository.murdoch.edu.au/25902/.
  • © 2013 British Educational Research Association.
  • Neurophysiological methods for monitoring brain activity in serious games and virtual environments: A review.
  • Authors: Manuel Ninaus, Silvia Erika Kober, Elisabeth V.C. Friedrich, Ian Dunwell, Sara de Freitas, Sylvester Arnab, Michela Ott, Milos Kravcik, Theodore Lim, Sandy Louchart, Francesco Bellotti, Anna Hannemann, Alasdair G. Thin, Riccardo Berta, Guilherme Wood, Christa Neuper.

Manuel Ninaus:

  • M.Sc., PhD-Student Department of Psychology, Section Neuropsychology University of Graz Universitätsplatz 2/III 8010 Graz Austria manuel.ninaus@uni-graz.at Manuel Ninaus is Research Assistant and PhD-candidate at the Department of Psychology (Section Neuropsychology) at the University of Graz, Austria.
  • Since 2010 he works as Research Assistant at the Department of Psychology and at the University of Graz and is involved in national and international scientific projects.
  • His research topics are neuronal plasticity through learning, EEG-based neurofeedback and auditory mirror neurons.
  • Furthermore Manuel Ninaus has many years of experience with different neurophysiological methods such as EEG, NIRS and fMRI.

Silvia Erika Kober:

  • Dr., Postdoctoral researcher Department of Psychology, Section Neuropsychology University of Graz Universitätsplatz 2/III 8010 Graz Austria silvia.kober@uni-graz.at.
  • Born in Graz, Austria, she received her MSc in Psychology from University of Graz, Austria in 2009 and obtained the Ph.D. degree from the University of Graz, Austria in 2012.
  • Since 2007 she works as Research Assistant at the Department of Psychology (Section Neuropsychology) at the University of Graz and is involved in national and international scientific projects.
  • Her main research interest is brain-computer interface (BCI).
  • She explored different mental tasks to control an EEGbased BCI, the impact of distraction on user performance as well as improvements of BCI usability for severely motor impaired individuals.

Ian Dunwell:

  • Dr., Postdoctoral researcher The Serious Games Institute Coventry University Technology Park Innovation Village Cheetah Road Coventry CV1 2TL United Kingdom IDunwell@cad.coventry.ac.uk.
  • Ian Dunwell is a senior researcher at the Serious Games Institute.
  • He holds a BSc in Physics from Imperial College London and a PhD in Computer Science from the University of Hull.
  • He has responsibility for a wide portfolio of health-related Serious Game research and development projects.
  • He is also the Technical Lead for the Applied Research Group.

Sara de Freitas:

  • Prof., Director of Research at the Serious Games Institute The Serious Games Institute Coventry University Technology Park Innovation Village Cheetah Road Coventry CV1 2TL United Kingdom s.defreitas@coventry.ac.uk.
  • Sara de Freitas is director of research at the Serious Games Institute, Professor of Virtual Environments and Fellow of the Royal Society of Arts.
  • Sara leads the applied research team at the Serious Games Institute and the Serious Games and Virtual Worlds Applied Research Group at Coventry University.

Sylvester Arnab:

  • Senior researcher, PhD The Serious Games Institute Coventry University Technology Park Innovation Village Cheetah Road Coventry CV1 2TL United Kingdom s.arnab@coventry.ac.uk.
  • He holds a doctorate from The University of Warwick and he is coordinating the SGI's contribution to the EU Funded Games and Learning Alliance (GALA).

Michela Ott:

  • Dr., Researcher Istituto Tecnologie Didattiche (ITD) Consiglio Nazionale delle Ricerche (CNR) Via De Marini, 6 16149 Genova Italy ott@itd.cnr.it Michela.
  • Ott is a senior researcher at the Institute for Educational Technology of Italian National Research Council.
  • She carries out research in the fields of: cognitive processes underpinning learning, educational use of software tools (with a special focus on Serious Games), special education, einclusion, learning design, distance education.
  • She is the author of more than 150 scientific publications.
  • At present she is the coordinator of the MAGICAL (MAking Games in CollaborAtion for Learning- KA3 LLP project), is involved in the eSG LLP project (Stimulating Entrepreneurship through Serious games-Erasmus FEXI) and in three European networks: GaLA (Game and Learning Alliance); V-MuST.net (Virtual Museum Transnational Network) and ETNA (European Thematic Network on Assistive Information and Communication Technologies).

Milos Kravcik:

  • Dr., Research Associate Informatik 5 – Information Systems RWTH Aachen University Ahornstr. 55 52056 Aachen Germany kravcik@dbis.rwth-aachen.de.
  • Milos Kravcik has a diploma degree in computer science and a doctoral degree in applied informatics from the Comenius University in Slovakia.
  • He co-organized several TEL doctoral schools and serves also as executive peer-reviewer or editorial board member for several journals related to learning technologies.
  • Theodore Lim is an active member within the Digital Tools Group; part of the EPSRC-funded Innovative Manufacturing Research Centre (IMRC) at Heriot-Watt University (www.smi.hw.ac.uk).
  • With over 40 international publications, a book and the successful commercialisation of his novel feature recognition algorithms, he is now applying his knowledge and expertise to the domain of serious games, game ware and computational biometrics for next generation engineering applications.

Sandy Louchart:

  • Dr, Lecturer School of Mathematical & Computer Sciences; Computer Science EM1.38, Heriot-Watt University Edinburgh EH14 4AS United Kingdom s.louchart@hw.ac.uk.
  • Sandy Louchart is a lecturer in Human Computer Interaction and Interaction Design in the School of Mathematical and Computer Sciences (MACS) at Heriot-Watt University.
  • His PhD, awarded by the University of Salford in 2007, explored the domain of Interactive Storytelling (IS) via the development of the Emergent Narrative concept.
  • The work was conducted in the domains of Artificial Intelligence, Synthetic Characters and Interactive Narratives and involved the design of Autonomous Synthetic Characters and their affect-based action/selection mechanisms to simulate characterisation within an interactive drama scenario.
  • His work has been published internationally in Intelligent Agents, Virtual Reality, Game and Interactive Storytelling Journals and Conferences.

Francesco Bellotti:

  • Assistant Professor, Dr. ELIOS Lab; Department of Naval, Electrical, Electronic and Telecommunications Engineering University of Genoa Via Opera Pia 11a 16145 Genova Italy franz@elios.unige.it.
  • Francesco Bellotti is assistant professor at the Department of Naval, Electrical, Electronic and Telecommunications Engineering of the University of Genoa, currently teaching Object Oriented Programming and Databases.
  • His main research interests are in the field of serious games, artificial intelligence, user modeling, neuroscientific signal analysis, technology enhanced learning and Human-Computer Interaction.
  • He has been the responsible of the design and implementation WPs of several European and Italian industrial research projects.
  • He authored 120+ papers in international journals and conferences.

Anna Hannemann:

  • Dipl. –Inform. Informatik 5 – Information Systems RWTH Aachen University Ahornstr. 55 52056 Aachen Germany hannemann@dbis.rwth-aachen.de.
  • Anna Hannemann is a PhD student at Information Systems of the RWTH Aachen University.
  • She holds a Diploma in Computer Science from RWTH Aachen University.
  • Within the EU-founded Network of Excellence for Serious Games (GALA, www.galanoe.eu), she works on serious gaming in context of neuroscience, community nurturing and interactive storytelling.
  • Her research interests include analysis of digital social networks and design of innovative social engineering and learning techniques.

Alasdair G. Thin:

  • Dr., Lecturer School of Life Sciences Heriot-Watt University Edinburgh EH14 4AS United Kingdom a.g.thin@hw.ac.uk.
  • He has a BSc in Physiology and Sports Science, Postgraduate Certificates in Information Technology and Academic Practice and a Doctorate in Clinical Exercise Physiology.
  • He first got interested in Serious Games at the beginning of 2004 after a chance encounter with some of the early EyeToy Play games for the Sony Playstation 2 that left him somewhat breathless!.
  • Having previous worked in health promotion, his research interests include how Serious Games can potentially act as a gateway to starting and maintaining a physically active lifestyle as well as wider health applications.

Riccardo Berta:

  • Assistant Professor, Dr. ELIOS Lab; Department of Naval, Electrical, Electronic and Telecommunications Engineering University of Genoa Via Opera Pia 11a 16145 Genova Italy berta@elios.unige.it.
  • Riccardo Berta is Assistant Professor at ELIOS Lab , University of Genoa.
  • He is teaching "Programming Fundamentals" (6 ECTS) in the Electronic and Information Technology Bachelor Course and "Multimedia Digital Entertainment" (10 ECTS) in the Electronic Engineering Master Course.
  • Associate Professor, Dr. Department of Psychology, Section Neuropsychology Karl-Franzens-Universität Graz Universitätsplatz 2/III 8010 Graz Austria guilherme.wood@uni-graz.at Guilherme Wood is Assistant Professor at the Department of Psychology at the University of Graz in Austria, also known as Guilherme Wood.
  • His research foci are neurofeedback, the impact of aging on cognition and brain function, neurocognitive aspects of number processing and the genetic correlates of cognitive and brain activity.

Christa Neuper:

  • Prof., Rector of the University of Graz Department of Psychology, Section Neuropsychology University of Graz Universitätsplatz 2/III 8010 Graz Austria christa.neuper@uni-graz.at Christa Neuper obtained the Ph.D. degree from the University of Graz , Austria in 1984.
  • During 1985-1986 she worked as Scientific Co-worker at the Department of Psychology at UniGraz.
  • Between 1990 and 1994 she was University Assistant at the Institute for Biomedical Engineering, Department of Medical Informatics, University of Technology Graz , Austria, and during 1994-2005 she worked as Senior Scientist at the Ludwig Boltzmann-Institute for Medical Informatics and Neuroinformatics, .
  • She was Assistant Professor at the Department of Psychology, UniGraz, where she has the rank of a full professor of ‘Applied Neuropsychology: Human-Computer Interface’ since 2005.
  • Her research topics are EEG-based brain-computer communication and neurofeedback, event-related brain oscillations and cognition, and sensorimotor processing in motor actions and motor imagery.

Keywords:

  • Neurophysiological methods; brain; serious games; games; virtual environments; virtual reality; NIRS; near infrared spectroscopy; EEG; electroencephalography; fMRI; functional magnetic resonance imaging; neuroscience; learning; game based learning.
  • Neuroscientific methods can be used to collect brain activity and then be used as feedback to the user, allowing the game to adapt to the user’s interactions and inputs.
  • An extensive examination of the scientific literature was undertaken.
  • Regarding the assessment of the efficacy of game-based approaches, action video games have been shown to enhance a wide variety of perceptual skills such as visual selective attention (Green and Bavelier, 2003) and spatial attention and mental rotation (Feng, Spence and Pratt, 2007).
  • The issue of how user inputs can be fed back into the game adaptivity is an emerging line of game-related research not only to provide efficacy but also as an adaptive element in HCI and games.

2.1. EEG

  • With EEG, electrical activity of the brain can be recorded non-invasively at the scalp surface, which reflects the summed potential of ionic currents across membranes of single cells, thus a direct method of measuring brain activity.
  • These stimuli can elicit ERPs in the EEG.
  • General effects of interaction with virtual environments on nervous system activity have already been explored in the 1990s.
  • Furthermore, EEG oscillations are useful for assessing attention, concentration, fatigue, and interest during playing games (Yamada, 1998).
  • By collaborative problem solving they learn in the first place to integrate and coordinate their collective knowledge.

2.2. fMRI

  • FMRI is a popular non-invasive neuroscientific method.
  • The information gained from fMRI can be used, for example, to improve the effectiveness of serious games or how games affect the player’s brain.
  • Klasen and colleagues (2012) argued that the activation in sensory and motor networks could underpin the central role of simulation for flow experience.
  • Williams and colleagues (2005) identified a number of clusters of additional activity in the joint attention settings in comparison to non-joint attention conditions.
  • Shams and Seitz (2008) advocated multisensory teaching approaches as mirroring more closely evolved learning processes, suggesting unisensory approaches are sub-optimal and that their selection is based in practicality rather than pedagogy.

2.3. fNIRS

  • FNIRS can be used to explore the functional activation of the human cerebral cortex.
  • As a result, deoxygenated haemoglobin decreases, whereas oxygenated haemoglobin increases in the active brain region.
  • This circumstance is the major source of the BOLD contrast as measured with fMRI (Huettel, Song, & McCarthy, 2009; Telkemeyer et al., 2011).
  • FNIRS applications in game-based learning: Many game studies used fNIRS to examine haemodynamic changes in frontal brain areas during playing video games.
  • The majority found a decrease in oxygenated-haemoglobin during gaming which might result from attention demand or task load from the video games (Izzetoglu et al., 2004; Matsuda and Hiraki, 2006; Nagamitsu et al., 2006).

2.4. Pros and cons of different neurophysiological methods

  • Different neurophysiological methods can be used for monitoring and assessing cognitive processes in serious games and virtual environments.
  • (Huppert, Hoge, Diamond, Franceschini, & Boas, 2006).
  • Compared to fMRI, which is locally bounded to the installation site, fNIRS and EEG are more flexible and portable systems.
  • Finally, it should be noted that the costs of the different neurophysiological methods are quite different.
  • If examining brain responses to fast events of a game, the EEG with its high temporal resolution should be the method of choice.

3. Discussion

  • One of the issues emerging from their review and on-going pilot studies is that there is a lack of a common methodological approach in the use and validity assessment of neuroscientific methods in serious games or games in general.
  • Many decisions needs to be taken before conducting neurophysiological measurements, therefore it is essential to have a clear experimental design and hypotheses for psychophysiological studies.
  • Especially for explorative studies, a simple game design with only few variables should be used to be able to clearly identify which brain processes are involved or which skills are needed during gaming.
  • Study designs under such a paradigm should focus on defining and understanding expertise at a task established through an understanding of the prior knowledge or measured performance of subjects.
  • An increased presence experience in a VE should foster the transfer of knowledge acquired in the virtual environment to corresponding real world behaviour (Slater et al., 1996).

4. Summary and Conclusions

  • The paper summarises some of the leading scientific research studies in the emerging field of neuroscience applied to gaming where EEG, fMRI and fNIRS are used to investigate the learner‘s brain activity during game play.
  • The findings indicate that there is a need to focus study designs in a particular way , as addressed in the discussion, to ensure that these are effective, focused on direct task performance and thereby narrowing the variables.
  • The review indicates that there are key benefits in using these neuroscientific techniques for developing serious games, particularly to provide better user feedback and allow designers to get a better understanding of the neurophysiological outcomes of the learner during learning periods and tasks.
  • The state of the art clearly shows the potential for using these devices.
  • Yet, there are ways to reach a balance between both things.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

MURDOCH RESEARCH REPOSITORY
This is the author’s final version of the work, as accepted for publication
following peer review but without the publisher’s layout or pagination.
http://dx.doi.org/10.1504/IJTEL.2014.060022
Ninaus, M., Kober, S.E., Friedrich, E.V.C., Dunwell, I., de Freitas,
S., Arnab, S., Ott, M., Kravcik, M., Lim, T., Louchart, S., Bellotti,
F., Hannemann, A., Thin, A.G., Berta, R., Wood, G. and Neuper,
C. (2014) Neurophysiological methods for monitoring brain
activity in serious games and virtual environments: a review.
International Journal of Technology Enhanced Learning,
6 (1). pp. 78-103.
http://researchrepository.murdoch.edu.au/25902/
© 2013 British Educational Research Association.
It is posted here for your personal use. No further distribution is permitted.

Neurophysiological methods for monitoring brain activity in serious games and virtual
environments: A review.
Authors: Manuel Ninaus, Silvia Erika Kober, Elisabeth V.C. Friedrich, Ian Dunwell, Sara de
Freitas, Sylvester Arnab, Michela Ott, Milos Kravcik, Theodore Lim, Sandy Louchart,
Francesco Bellotti, Anna Hannemann, Alasdair G. Thin, Riccardo Berta, Guilherme Wood,
Christa Neuper
Manuel Ninaus:
M.Sc., PhD-Student
Department of Psychology, Section Neuropsychology
University of Graz
Universitätsplatz 2/III
8010 Graz
Austria
manuel.ninaus@uni-graz.at
Manuel Ninaus is Research Assistant and PhD-candidate at the Department of Psychology (Section
Neuropsychology) at the University of Graz, Austria. He received his MSc in Psychology from
University of Graz, Austria in 2012. Since 2010 he works as Research Assistant at the Department of
Psychology and at the University of Graz and is involved in national and international scientific
projects. He is currently involved in the EU GALA Network of Excellence (www.galanoe.eu). His
research topics are neuronal plasticity through learning, EEG-based neurofeedback and auditory
mirror neurons. Furthermore Manuel Ninaus has many years of experience with different
neurophysiological methods such as EEG, NIRS and fMRI.
Silvia Erika Kober:
Dr., Postdoctoral researcher
Department of Psychology, Section Neuropsychology
University of Graz
Universitätsplatz 2/III
8010 Graz
Austria
silvia.kober@uni-graz.at
Silvia Erika Kober is a postdoctoral researcher at the Department of Psychology (Section
Neuropsychology) at the University of Graz, Austria. Born in Graz, Austria, she received her MSc in
Psychology from University of Graz, Austria in 2009 and obtained the Ph.D. degree from the
University of Graz, Austria in 2012. Since 2007 she works as Research Assistant at the Department of
Psychology (Section Neuropsychology) at the University of Graz and is involved in national and
international scientific projects. Her research topics are EEG/NIRS-based brain-computer
communication and neurofeedback, interacting in virtual realities, virtual reality as rehabilitation tool
and spatial cognition.
Elisabeth V.C. Friedrich:
Dr., Postdoctoral researcher
Department of Psychology, Section Neuropsychology

University of Graz
Universitätsplatz 2/III
8010 Graz
Austria
elisabeth.friedrich@uni-graz.at
Elisabeth V.C. Friedrich studied Psychology at the University of Graz, Austria, and conducted her
master thesis at the Laboratory of Neural Injury and Repair, Wadsworth Center, Albany, NY. She has
received her doctorate in natural science in 2012 at the University of Graz, Austria. Her main research
interest is brain-computer interface (BCI). She explored different mental tasks to control an EEG-
based BCI, the impact of distraction on user performance as well as improvements of BCI usability
for severely motor impaired individuals. She is currently involved in the EU GALA Network of
Excellence (www.galanoe.eu).
Ian Dunwell:
Dr., Postdoctoral researcher
The Serious Games Institute
Coventry University Technology Park
Innovation Village
Cheetah Road
Coventry
CV1 2TL
United Kingdom
IDunwell@cad.coventry.ac.uk
Ian Dunwell is a senior researcher at the Serious Games Institute. He holds a BSc in Physics from
Imperial College London and a PhD in Computer Science from the University of Hull. He has
responsibility for a wide portfolio of health-related Serious Game research and development projects.
He is also the Technical Lead for the Applied Research Group.
Sara de Freitas:
Prof., Director of Research at the Serious Games Institute
The Serious Games Institute
Coventry University Technology Park
Innovation Village
Cheetah Road
Coventry
CV1 2TL
United Kingdom
s.defreitas@coventry.ac.uk
Sara de Freitas is director of research at the Serious Games Institute, Professor of Virtual
Environments and Fellow of the Royal Society of Arts. Sara leads the applied research team at the
Serious Games Institute and the Serious Games and Virtual Worlds Applied Research Group at
Coventry University.
Sylvester Arnab:
Senior researcher, PhD
The Serious Games Institute
Coventry University Technology Park
Innovation Village

Cheetah Road
Coventry
CV1 2TL
United Kingdom
s.arnab@coventry.ac.uk
Sylvester Arnab is a senior researcher at the SGI. He holds a doctorate from The University of
Warwick and he is coordinating the SGI's contribution to the EU Funded Games and Learning
Alliance (GALA).
Michela Ott:
Dr., Researcher
Istituto Tecnologie Didattiche (ITD)
Consiglio Nazionale delle Ricerche (CNR)
Via De Marini, 6
16149 Genova
Italy
ott@itd.cnr.it
Michela Ott is a senior researcher at the Institute for Educational Technology of Italian National
Research Council. She carries out research in the fields of: cognitive processes underpinning learning,
educational use of software tools (with a special focus on Serious Games), special education, e-
inclusion, learning design, distance education. She is the author of more than 150 scientific
publications. She is also the author of educational software packages and hypermedia systems. She
has participated in, led and coordinated a number of national and international research projects in the
field of Educational Technology. At present she is the coordinator of the MAGICAL (MAking Games
in CollaborAtion for Learning- KA3 LLP project), is involved in the eSG LLP project (Stimulating
Entrepreneurship through Serious games-Erasmus FEXI) and in three European networks: GaLA
(Game and Learning Alliance); V-MuST.net (Virtual Museum Transnational Network) and ETNA
(European Thematic Network on Assistive Information and Communication Technologies).
Milos Kravcik:
Dr., Research Associate
Informatik 5 Information Systems
RWTH Aachen University
Ahornstr. 55
52056 Aachen
Germany
kravcik@dbis.rwth-aachen.de
Milos Kravcik has a diploma degree in computer science and a doctoral degree in applied informatics
from the Comenius University in Slovakia. He has been dealing with Technology Enhanced Learning
(TEL) since 1988 in various national and international projects, later also at the Fraunhofer Institute
for Applied Information Technology in Germany and at the Open University in the Netherlands.
Since 2010 he works as a Research Fellow at the RWTH Aachen University and his main research
interests include personalized learning environments and self-regulated learning. He co-organized
several TEL doctoral schools and serves also as executive peer-reviewer or editorial board member
for several journals related to learning technologies.
Theodore Lim:

Dr., Lecturer
School of Engineering & Physical Sciences; Mechanical Engineering
NS G.04, Heriot-Watt University
Edinburgh
EH14 4AS
United Kingdom
t.lim@hw.ac.uk
Theodore Lim is an active member within the Digital Tools Group; part of the EPSRC-funded
Innovative Manufacturing Research Centre (IMRC) at Heriot-Watt University (www.smi.hw.ac.uk).
As both an academic and researcher with considerable industrial experience, he has been instrumental
in the research, analysis and development of virtual engineering environments in a variety of product
engineering domains and now focuses his work on the acquisition of engineering knowledge
information management systems within all aspect of product engineering; with a particular emphasis
on conceptual design. He has also implemented game-based learning methods in design and
manufacturing taught courses. With over 40 international publications, a book and the successful
commercialisation of his novel feature recognition algorithms, he is now applying his knowledge and
expertise to the domain of serious games, game ware and computational biometrics for next
generation engineering applications.
Sandy Louchart:
Dr, Lecturer
School of Mathematical & Computer Sciences; Computer Science
EM1.38, Heriot-Watt University
Edinburgh
EH14 4AS
United Kingdom
s.louchart@hw.ac.uk
Sandy Louchart is a lecturer in Human Computer Interaction and Interaction Design in the School of
Mathematical and Computer Sciences (MACS) at Heriot-Watt University. His PhD, awarded by the
University of Salford in 2007, explored the domain of Interactive Storytelling (IS) via the
development of the Emergent Narrative concept. The work was conducted in the domains of Artificial
Intelligence, Synthetic Characters and Interactive Narratives and involved the design of Autonomous
Synthetic Characters and their affect-based action/selection mechanisms to simulate characterisation
within an interactive drama scenario. The research led to the development of a novel approach to
Synthetic Character action/selection mechanisms featuring projection-based affective planning so as
to select dramatically intense actions and events for interaction; the Double Appraisal action/selection
mechanism. His work has been published internationally in Intelligent Agents, Virtual Reality, Game
and Interactive Storytelling Journals and Conferences.
Francesco Bellotti:
Assistant Professor, Dr.
ELIOS Lab; Department of Naval, Electrical, Electronic and Telecommunications Engineering
(DITEN)
University of Genoa
Via Opera Pia 11a
16145 Genova
Italy
franz@elios.unige.it

Citations
More filters

Michela Ott1, Laura Freina1Institutions (1)
01 Jan 2015
TL;DR: A survey on the scientific literature on the advantages and potentials in the use of Immersive Virtual Reality in Education in the last two years shows how VR in general, and immersive VR in particular, has been used mostly for adult training in special situations or for university students.
Abstract: Since the first time the term "Virtual Reality" (VR) has been used back in the 60s, VR has evolved in different manners becoming more and more similar to the real world. Two different kinds of VR can be identified: non-immersive and immersive. The former is a computer-based environment that can simulate places in the real or imagined worlds; the latter takes the idea even further by giving the perception of being physically present in the non-physical world. While non-immersive VR can be based on a standard computer, immersive VR is still evolving as the needed devices are becoming more user friendly and economically accessible. In the past, there was a major difficulty about using equipment such as a helmet with goggles, while now new devices are being developed to make usability better for the user. VR, which is based on three basic principles: Immersion, Interaction, and User involvement with the environment and narrative, offers a very high potential in education by making learning more motivating and engaging. Up to now, the use of immersive-VR in educational games has been limited due to high prices of the devices and their limited usability. Now new tools like the commercial "Oculus Rift", make it possible to access immersive-VR in lots of educational situations. This paper reports a survey on the scientific literature on the advantages and potentials in the use of Immersive Virtual Reality in Education in the last two years (2013-14). It shows how VR in general, and immersive VR in particular, has been used mostly for adult training in special situations or for university students. It then focuses on the possible advantages and drawbacks of its use in education with reference to different classes of users like children and some kinds of cognitive disabilities (with particular reference to the Down syndrome). It concludes outlining strategies that could be carried out to verify these ideas.

481 citations


Cites background from "Neurophysiological methods for moni..."

  • ...The feeling of being somewhere real when you are in VR is well known to researchers, and is referred to as “presence,” [42] and it most distinguishes VR from 3D on a screen [43]....

    [...]


Journal ArticleDOI
28 May 2015
TL;DR: It is concluded that IA may pose a serious health hazard to a minority of people and there is no consensual definition for IA, with implications for treatment, prevention, and patient-specific considerations.
Abstract: Research into Internet addiction (IA) has grown rapidly over the last decade The topic has generated a great deal of debate, particularly in relation to how IA can be defined conceptually as well as the many methodological limitations The present review aims to further elaborate and clarify issues that are relevant to IA research in a number of areas including: definition and characterization, incidence and prevalence rates, associated neuronal processes, and implications for treatment, prevention, and patient-specific considerations It is concluded that there is no consensual definition for IA Prevalence rates among nationally representative samples across several countries vary greatly (from 1% to 187%), most likely reflecting the lack of methodological consistency and conceptual rigor of the studies The overlaps between IA and other more traditional substance-based addictions and the possible neural substrates implicated in IA are also highlighted In terms of treatment and prevention, both psychological and pharmacological treatments are examined in light of existing evidence alongside particular aspects inherent to the patient perspective Based on the evidence analyzed, it is concluded that IA may pose a serious health hazard to a minority of people

143 citations


Cites background from "Neurophysiological methods for moni..."

  • ...Internet addiction and its associated neuronal processes Over the last 15 years, studies have emerged using neuroscientific techniques to study relevant brain processes, activities, and brain structures associated with both gaming and IA.(79,80) Neuroimaging studies have a number of advantages over the self-reports that historically have more commonly been used in IA research....

    [...]


Journal ArticleDOI
TL;DR: It is argued that research on flow should focus on the specific aspects related to the very nature of serious games that combine enjoyment and learning, and new methods to measure flow and analyse the data need to be developed and studied.
Abstract: The entertaining elements implemented in a serious game are key factors in determining whether a player will be engaged in a play-learn process and able to achieve the desired learning outcomes. Thus, optimization of subjective playing experience is a crucial part of a game design process. Flow theory can be adopted for measuring user experience and analyzing the quality of serious game designs. In addition, flow seems to have a positive influence on performance enhancement, learning and engagement. The focus of this review is especially on examining the meaning of flow in the context of serious games as well as exploring the relationship between flow and learning, factors that influence occurrence of flow and how flow is operationalized. The review revealed that there are mainly conceptual considerations about flow in serious games, but no robust empirical evidence about the meaning of flow. This is in line with other studies. We argue that research on flow should focus on the specific aspects related to the very nature of serious games that combine enjoyment and learning. Furthermore, new methods to measure flow and analyse the data need to be developed and studied.

55 citations


Cites background from "Neurophysiological methods for moni..."

  • ...The combination of subjective and objective measurements can increase the explained variance of player experiences such as flow, immersion and engagement [25, 28]....

    [...]

  • ...Advances in neurosciences are making it possible to continuously monitor the player status [25, 26]....

    [...]


Journal ArticleDOI
Michel Wedel1, Enrique Bigné2, Jie Zhang1Institutions (2)
TL;DR: A conceptual framework for VR/AR research in consumer marketing is proposed that centers around consumer experiences provided byVR/AR applications along the customer journey and the effectiveness of such applications, and delve into the key concepts and components of the framework.
Abstract: Virtual reality (VR) and augmented reality (AR) technologies are having a profound impact on a variety of marketing practices and are attracting increasing attention from marketing researchers In this article, we review developments in VR/AR applications and research in the area of consumer marketing We propose a conceptual framework for VR/AR research in consumer marketing that centers around consumer experiences provided by VR/AR applications along the customer journey and the effectiveness of such applications, and delve into the key concepts and components of the framework Next, we provide a comprehensive overview of VR/AR applications in current practices and extant research on VR/AR in consumer marketing Finally, based on this framework, we offer an outlook for future developments of VR/AR technologies and applications, discuss managerial implications, and prescribe directions for research on consumer marketing

43 citations


Book ChapterDOI
01 Jan 2017
TL;DR: This chapter describes how multimodal serious games can create an immersive experience to enhance the visitor’s experience and a framework for multi-modal cultural heritage is proposed based on the above technologies.
Abstract: This chapter describes how multimodal serious games can create an immersive experience to enhance the visitor’s experience. The creation of more engaging digital heritage exhibitions by seamlessly integrating technologies to provide a multimodal virtual and augmented reality experience is presented. In engaging exhibitions, participants can switch between different modes of exploring the physical artefacts at the museum and can explore these artefacts further through serious games, user interfaces, virtual reality and augmented reality. Different types of interaction paradigms are also illustrated. Moreover, a framework for multimodal cultural heritage is proposed based on the above technologies. Finally, future research directions for creating new opportunities for scientific research are presented.

42 citations


References
More filters

Journal ArticleDOI
Giacomo Rizzolatti1, Laila Craighero2Institutions (2)
TL;DR: A neurophysiological mechanism appears to play a fundamental role in both action understanding and imitation, and those properties specific to the human mirror-neuron system that might explain the human capacity to learn by imitation are stressed.
Abstract: A category of stimuli of great importance for primates, humans in particular, is that formed by actions done by other individuals. If we want to survive, we must understand the actions of others. Furthermore, without action understanding, social organization is impossible. In the case of humans, there is another faculty that depends on the observation of others' actions: imitation learning. Unlike most species, we are able to learn by imitation, and this faculty is at the basis of human culture. In this review we present data on a neurophysiological mechanism--the mirror-neuron mechanism--that appears to play a fundamental role in both action understanding and imitation. We describe first the functional properties of mirror neurons in monkeys. We review next the characteristics of the mirror-neuron system in humans. We stress, in particular, those properties specific to the human mirror-neuron system that might explain the human capacity to learn by imitation. We conclude by discussing the relationship between the mirror-neuron system and language.

6,351 citations


"Neurophysiological methods for moni..." refers background in this paper

  • ...…simulating the actions of others and might be the basis for social processes, such as imitation, empathy, action understanding and perspective taking (Rizzolatti and Craighero, 2004) – have shown different EEG activation in conditions in which participants performed synchronised finger movements…...

    [...]


Journal ArticleDOI
TL;DR: Quantification of ERD/ERS in time and space is demonstrated on data from a number of movement experiments, whereby either the same or different locations on the scalp can display ERD and ERS simultaneously.
Abstract: An internally or externally paced event results not only in the generation of an event-related potential (ERP) but also in a change in the ongoing EEG/MEG in form of an event-related desynchronization (ERD) or event-related synchronization (ERS). The ERP on the one side and the ERD/ERS on the other side are different responses of neuronal structures in the brain. While the former is phase-locked, the latter is not phase-locked to the event. The most important difference between both phenomena is that the ERD/ERS is highly frequency band-specific, whereby either the same or different locations on the scalp can display ERD and ERS simultaneously. Quantification of ERD/ERS in time and space is demonstrated on data from a number of movement experiments.

5,454 citations


"Neurophysiological methods for moni..." refers background or methods in this paper

  • ...These stimuli can elicit ERPs in the EEG....

    [...]

  • ...A recent study by Kober and Neuper (2012) showed that ERPs in EEG can be used as a valuable and objective method for measuring the presence in VEs....

    [...]

  • ...Additionally, during the navigation task cortical auditory ERPs were elicited by releasing footstep sounds binaurally as they moved forward in the virtual world....

    [...]

  • ...By averaging these events, ERPs can be detected (Pfurtscheller and Lopes, 1999)....

    [...]

  • ...These signals can be differentiated in two groups: first, the EventRelated Potentials (ERPs), such as P300 evoked potentials....

    [...]


Book
12 Dec 2000
Abstract: Research published by University of Rochester neuroscientists C. Shawn Green and Daphne Bavelier has grabbed national attention for suggesting that playing "action" video and computer games has the positive effect of enhancing student's visual selective attention. But that finding is just one small part of a more important message that all parents and educators need to hear: Video games are not the enemy, but the best opportunity we have to engage our kids in real learning.

3,255 citations


Book
01 Apr 1993
Abstract: The main thrust of Electroencephalography is to preserve the sound basis of classic EEG recording and reading and, on the other hand, to present the newest developments for future EEG/neurophysiology research, especially in view of the highest brain functions. The Fourth Edition features new chapters on modern and future oriented EEG/EP research, spinal monitoring and dipole modelling

3,194 citations


01 Jan 1998
TL;DR: Historical aspects introduction to the neurophysiological basis of the EEG and DC potentials cellular substrates of spontaneous and evoked brain rhythms dynamics of EEG as signals and neuronal populations are introduced.
Abstract: Historical aspects introduction to the neurophysiological basis of the EEG and DC potentials cellular substrates of spontaneous and evoked brain rhythms dynamics of EEG as signals and neuronal populations - models and theoretical considerations biophysical aspects of EEG and magnetoencephalogram generation technological basis of the EEG recording EEG recording and operation of the apparatus the EEG signal - polarity and field determination digitized (paperless) EEG recording the normal EEG in the waking adult sleep and EEG maturation of the EEG - development of waking and sleep patterns EEG patterns and genetics nonspecific abnormal EEG patterns abnormal EEG patterns - epileptic and paroxysmal activation methods brain tumours and other space-occupying lesions (with a section on oncological CNS complications) the EEG in cerebral inflammatory processes cerebrovascular disorders and EEG EEG and old age EEG and dementia EEG and neurodegenerative disorders the EEG in infantile brain damage and cerebral palsy craniocerebral trauma metabolic central nervous system disorders cerebral anoxia - experimental view cerebral anoxia - clinical aspects coma and brain death epileptic seizure disorders non-epileptic attacks polygraphy polysomnography - principles and applications in sleep and arousal disorders neonatal electroencephalography event-related potentials - methodology and quantification contingent negative variation and Bereitschafts-potential visual evoked potentials auditory evoked potentials evoked potentials in infancy and childhood neurometric use of event-related potentials event-related potentials - P 300 and psychological implications neuroanaesthesia and intraoperative neurological monitoring clinical use of magnetoencephalography brain mapping - methodology the clinical use of brain mapping EEG analysis - theory and practice the EEG in patients with migraine and other headaches psychiatric disorders and the EEG psychology, physiology and the EEG EEG in aviation, space exploration and diving EEG and neuropharmacology - experimental approach EEG, drug effect and central nervous system poisoning toxic encephalography the special form of stereo-electroencephalography electroencephalography subdural EEG recordings special techniques of recording and transmission prolonged EEG monitoring in the diagnosis of seizure disorders EEG monitoring during carotid endarterectomy and open heart surgery computer analysis and cerebral maturation special use of EEG computer analysis in clinical neurology.

3,113 citations


"Neurophysiological methods for moni..." refers background in this paper

  • ...38, Edinburgh, EH14 4AS, UK Email: s.louchart@hw.ac.uk...

    [...]

  • ...These EEG oscillations can shed light on cognitive and emotional processes underlying the gaming process (Niedermeyer and Lopes da Silva, 2005)....

    [...]


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
20214
20205
20195
20184
20179
20162