![Fig. 4 Technical flowchart for video see-through augmented reality on mobile devices [32]](/figures/fig-4-technical-flowchart-for-video-see-through-augmented-6wxlw3c2.webp)
A Study of Developments and Applications of Mixed Reality
Cubicles and Their Impact on Learning
J.
Uhomoibhi
Artificial Intelligence Research Group, Ulster University, Northern Ireland, UK
C.
Onime
International Centre for Theoretical Physics, Trieste, Italy
H.
Wa
ng
Artificial Intelligence Research Group, Ulster University, Northern Ireland, UK
Abstract
Purpose - This paper reports on developments and applications of mixed reality cubicles and their
impacts on learning in higher education. This paper investigates and presents the cost effective
application of augmented reality (AR) as a mixed reality technology via or to mobile devices such as
head-mounted devices, smart phones and tablets. Discuss the development of mixed reality applications
for mobile (smartphones and tablets) devices leading up to the implementation of a mixed reality cubicle
for immersive three dimensional (3D) visualizations.
Design/methodology/approach - The approach adopted was to limit the considerations to the application
of AR via mobile platforms including head-mounted devices with focus on smartphones and tablets,
which contain basic feedback –to-user channels such as speakers and display screens. An AR
visualization cubicle was jointly developed and applied by three collaborating institutions. The markers,
acting as placeholders acts as identifiable reference points for objects being inserted in the mixed reality
world. Hundreds of participants comprising academics and students from seven different countries took
part in the studies and gave feedback on impact on their learning experience.
Findings - Results from current study show less than 30% had used mixed reality environments. This is
lower than expected. About 70% of participants were first time users of mixed reality technologies. This
indicates a relatively low use of mixed reality technologies in education. This is consistent with research
findings reported that educational use and research on augmented reality is still not common despite their
categorization as emerging technologies with great promise for educational use.
Research limitations/implications - Current research has focused mainly on cubicles which provides
immersive experience if used with head-mounted devices (goggles and smartphones), that are limited by
their display/screen sizes. There are some issues with limited battery lifetime for energy to function,
hence the need to use rechargeable batteries. Also, the standard dimension of cubicles does not allow for
group visualizations. The current cubicle has limitations associated with complex gestures and
movements involving two hands, as one hand are currently needed for holding the mobile phone.
Practical implications - The use of mixed reality cubicles would allow and enhance information
visualization for big data in real time and without restrictions. There is potential to have this extended for
use in exploring and studying otherwise inaccessible locations such as sea beds and underground caves.
Social implications - Following on from this study further work could be done to developing and
application of mixed reality cubicles that would impact businesses, health, and entertainment.
Originality/value - The originality of this paper lies in the unique approach used in the study of
developments and applications of mixed reality cubicles and their impacts on learning. The diverse
composition in nature and location of participants drawn from many countries comprising of both tutors
and students adds value to the present study. The value of this research include amongst others, the useful
results obtained and scope for developments in the future.
Keywords Mixed Reality, Cubicles, CAVE, Mobile Computing, Learning impacts
Paper type Research paper
1. Introduction
Humans typically perceive and relate with their surrounding environment using the five
physiological senses of sight, smell, touch, sound and taste, although sight, sound and
touch are more readily used. Mixed Reality technology has the potential to offer richer
information, increase learner engagement and to improve the educational offering for different
categories of learners. Augmented reality as the leading technology has the capability to engage
the user in an enhanced perception of the surroundings as well as the possibility to act as a bridge
towards different types of contents encompassing text, audio, video. AR is characterized by the
combination of real and virtual components and by interaction in real time (Azuma, 1997;
Milgram and Kishino, 1994;).
The portability of technology over the years, has seen a shift from the use of heavy backpaks
and associated displays to the use of light glasses connected to mobile devices such as Google
Glass (Google Glass, 2013) and the futuristic AR contact lense. Most recently, the use of
Microsoft's Hololens platform (Microsoft Hololens) has resulted in new levels of immersion to a
holographic AR reality experience, with the help of a head-mounted display embedding all the
hardware (Cheok et al, 2004; M. Ostanin, A. Klimchik, 2018; Lang et al, 2019).
Reality may be considered as a state of having existence, substance or objects that may be
actually experienced and/or seen (Onime and Abiona, 2016), while virtuality may be considered
as having a non-realistic (or abstract) view of objects, that is opposite of an idealistic,
realistic or notional view. This opposing relationship between reality on the one hand and
virtuality on the other hand is illustrated in Figure 1, where reality is at one extreme of a
continuum while virtuality, better known as Virtual Reality (VR), is at the opposite extreme
and in-between them is the mixed-reality environment (Onime et. al., 2016).
Fig.
Real it y
-Virtuality Continuum. Adapted
from (
Milgram et. al. 1994)
Traveling a l o ng the
continuum
from left to
righ
t
represents diminishing
reality (or
reduction
in real objects) and increasing
virtuality
(increase
in
virtual objects) resulting
in the complete absence of real objects at the
virtual
end. In other words, at the VR end,
the
environmen
t
is completely made
up
of virtual objects. Two kinds of mixed reality
environments
are
presen
t
in
the
continuum: Augmented
Reality (AR), where the
environmen
t
is
predomin
antly
composed of real objects and
Augmented Virtuality
(AV)
where it is made
up
of virtual o b j e c t s . In AR, the goal is not to exclude the real
objects (as
in
VR) but to blend
additional
or
computer generated information
into the
real
world. While in AV, the goal is to blend real objects
(data
or
information
from
real world) into a
computer generated
environmen
t
(Onime et. al., 2016)
. From Figure
1,
it is not difficult to imagine a centroid
p
oin
t
of the
continuum
where it is
no
longer
possible to
distinguish
the real world from the virtual world (Milgram et. al. 1994), loca
ted
hypothetically
between AR and AV
that
represen
t
a
situation
of balance,
or
equal
number of real and virtual
objects,
In general, the
environmen
t
described by the
continuum
may be simplified as the
integration
of real and virtual objects as shown in
Equation 1.
E
(1)
Where E represents
the
environment,
R the set of real objects and
V
the
set
of virtual
objects.
As earlier discussed, E may be
conditionally
grouped into
distinct
environ
ments as
follows:
(2)
Where
E
R
,
E
AR
, E
c
, E
AV
and E
V
R
represen
t
the Real, AR, centroid,
AV
and VR
environments
respectively, each of which may be
individually
expanded from
Equation
1.
Eliminating
the
extremities
from
Equation
2 then results in the
mixed
reality
e
n
vironmen
t
as show in
Equation 3.
(3)
Where,
E
M
R
represents
the mixed reality
environmen
t.
In practice, the solution of
Equations
3 and 1 is simplified during
the
creation of mixed
reality
environments
by
introducing
/
using special
place
holders known as markers to
indicate the relative
entry-points
(or
p
ositions)
and/or orientation
of other (to be
introduced)
objects within
environment.
F
or
example, in the visual form of AR, the marker
is a graphically visible
image
that
should be recognised at
run-time
from
d
iffe
ren
t
distances,
resolutions
and
angles.
That is,
(4)
Where R
p
is the set of real place-holders used for insertion of virtual
objects
(5)
Where
V
p
is the set of virtual place-holders used for insertion of real
objects.
E
c
is simply the
special case of either
Equation
4 or 5 when the
cardinalit
y
of both sets (under the integral
sign) are
e
q
ual.
1.1 Virtual Reality
(VR)
A broad definition of VR
portrays
it as a technology
that attempts
to
pro
vide
3D
interactions
with a
computer
in new ways with emphasis on the
heigh
t
ened use of the
human senses of sight, sound and touch. For example,
spa
tialized sound may be used to
provide direction such as sound growing
louder
as the user approaches (Zahorik, 2002). While,
a narrower definition describes VR as
a
3D
computer-generated
simulation oriented
environmen
t
that
allows users
to
interact
at various levels in a more
natural
manner using
interface devices
and
peripherals such as 3D eye-wear and trackers [9]. For example, haptic
devices allow users (with a VR
environment)
to touch surfaces, grasp and move
virtual
objects, possibly
obtaining feedback/reactions
them (Basdogan et. al., 2000; Tan and Pentland,
1997).
In VR, the user undergoes an immersion or the psychological
exp
erience
of loosing
himself in the
computer (digitally) generated
environmen
t
(virtual
space or world)
that
may be sometimes modeled after or based on an
existing
(real)
environment.
Although, in
such virtual world(s), everything is possible
as
t
ypical
laws of physics such as gravity and
time may be modified or
eliminated
completely and the users can (within its confines)
overcome
limitations
that
were previously imposed by the physical world (Loscos et. al.
2003).
VR h a s b e e n c l a s s i f i ed into non, semi and fully immersive systems,
according
to
the degree of immersion experienced by the users (Fox et. al., 2009). In non-immersive
VR
systems, users do not have a stereo view
and/or
experience of the virtual
en
vi
ronment.
Semi-immersive VR systems provide a bigger view of the
computer generated
environmen
t
mainly
through
use of a large screen device or
sp
ecial eye-wear (or goggles), commonly
combined with special input devices such
as
wands, gloves or controllers.
Fully-immersive
VR systems provide a total
(3D)
view of the
computer generated
environmen
t
obtained
using multiple
large
screen devices or special eye-wear along with special input devices
such
as touch-screens,
wands, gloves and
con
trollers.
Figure 2 shows
t
wo
differen
t
examples of VR
environments,
the first
rep-
resents an
indoor
environmen
t
with various bits of
furniture
including
c
hairs,
a sofa and a
painting,
while the second is an outdoor view of a well
dev
elop
ed
w
ater-fron
t.
In many VR systems as discussed in the literature review section 2, full immersion
occurs
when
all references to the real world
environmen
t
are completely removed by
housing
the user in specially designed CAVE
environments(s)
or using special
head-
mounted
displays (HMD) (helmet devices with mounted displays) for
mobility
.
This paper discusses
obtaining
similar heightened (fully) immersive
exp
erience
using
mixed-reality
technology
and Section 3 presents the
development, lim
itations
of a fully immersive
mixed-reality
cubicle and results of a study
on
familiarity with mixed reality technologies at
t
wo
diffe
r
e
n
t
academic
institu
tions, while Section 4 concludes the
pap
er.
Fig.
2
Examples
of VR
environments. (Santa’s Company, 2013)
2. Literature Review
VR and mixed-reality are two technologies that are changing the future directions of
ubiquitous computing and there are already, many diverse applications of VR technology in
various sectors: For example, VR has been used as a plat- form to study differences in human
behaviour within controlled environment and the real physical world (Santa’s Company, 2013).
It has also been used as a platform for teaching specialized procedures to pilots (Pausch et. al.,
1992) and doctors (O’Toole et. al, 1998) without the associated risks involved in a real
environment.
Mixed reality applications now surround us everywhere in education, at home and in industry.
They are most obviously in video games and entertainment, but also in live events, in retail,
education, healthcare and engineering (Quint, 2015; Bellini et al, 2016; The Ford Motor
Company, 2017). They are used for information visualization, remote collaboration, human-
machine-interfaces, design tools as well as education and training (Scholz and Smith, 2016;
Bacca et al, 2015).
Mixed Reality combines the real world and the virtual world into one user experience, which
significantly helps to extend opportunities for enhanced real learning (Lee, 2012; Guo, 2015). In
the face of rapid technological developments with increasing student number abd diversity of
needs, there is the search for new ways to teach, it could be argued that AR has the potential
pedagogical applications to meet some of the needs.
In the education sector, there are on-line resources that use non-immersive VR related
techniques to provide several chemistry laboratory experiments/exercises , as well as, simulation
of a chemistry laboratory through use of rich media powered by JavaScript (Georgiou et. al,
2007). In civil engineering, building technology and architecture, VR based prototyping is also
commonly used to provide a 3D view (or 3D printed model) of objects with varying levels of
abstraction (Cecil and Huber, 2010).
Virtual reality is used to provide the interactive display of 3D objects in the gaming industry