A study of developments and applications of mixed reality cubicles and their impact on learning
Summary (4 min read)
1. Introduction
- Humans typically perceive and relate with their surrounding environment using the five physiological senses of sight, smell, touch, sound and taste, although sight, sound and touch are more readily used.
- Augmented reality as the leading technology has the capability to engage the user in an enhanced perception of the surroundings as well as the possibility to act as a bridge towards different types of contents encompassing text, audio, video.
- This opposing relationship between reality on the one hand and virtuality on the other hand is illustrated in Figure 1, where reality is at one extreme of a continuum while virtuality, better known as Virtual Reality (VR), is at the opposite extreme and in-between them is the mixed-reality environment (Onime et. al., 2016).
- In AR, the goal is not to exclude the real objects (as in VR) but to blend additional or computer generated information into the real world.
- In practice, the solution of Equations 3 and 1 is simplified during the creation of mixed reality environments by introducing / using special place holders known as markers to indicate the relative entry-points (or positions) and/or orientation of other (to be introduced) objects within environment.
1.1 Virtual Reality (VR)
- Spatialized sound may be used to provide direction such as sound growing louder as the user approaches (Zahorik, 2002).
- Haptic devices allow users (with a VR environment) to touch surfaces, grasp and move virtual objects, possibly obtaining feedback/reactions them (Basdogan et. al., 2000; Tan and Pentland, 1997).
- In such virtual world(s), everything is possible as typical laws of physics such as gravity and time may be modified or eliminated completely and the users can (within its confines) overcome limitations that were previously imposed by the physical world (Loscos et. al. 2003).
- In non-immersive VR systems, users do not have a stereo view and/or experience of the virtual environment.
- Fully-immersive VR systems provide a total (3D) view of the computer generated environment obtained using multiple large screen devices or special eye-wear along with special input devices such as touch-screens, wands, gloves and controllers.
2. Literature Review
- It has also been used as a platform for teaching specialized procedures to pilots (Pausch et. al., 1992) and doctors (O’Toole et. al, 1998) without the associated risks involved in a real environment.
- They are used for information visualization, remote collaboration, humanmachine-interfaces, design tools as well as education and training (Scholz and Smith, 2016; Bacca et al, 2015).
- Virtual reality is used to provide the interactive display of 3D objects in the gaming industry (massive on-line role playing games) and scientific research work especially those involving modeling and simulation.
- Mixed reality technology provides the opportunities to combine learning and entertainment in new ways especially suited for laboratory and classroom (Davidsson et. al. 2012).
2.1 Cave Automatic Virtual Environment (CAVE)
- In most implementations of CAVEs, the walls (including floor and ceiling) are replaced by large (wall-sized) displays or projection screens arranged such that the computer generated environment is projected all around the user.
- Within CAVEs, VR systems also have to track and respond to, the user’s physical orientation, movements and gestures.
- Sometimes, this may involve the use of special hand-gloves or body suits suitable for tracking movements in very fine detail.
- Another example is the Wall-sized Interaction with Large Datasets (WILD) room (Beaudouin-Lafon et. al., 2012).
- In WILD, the CAVE room could be used by a group of microbiologist (co-located inside the CAVE) to study how one molecule docks with another and interactively and seamlessly switch between several 3D representations, different molecular models, online databases, websites and research articles along with the ability to collaborate with colleagues in remote locations (Beaudouin-Lafon et. al., 2012).
2.2 Immersive Mixed Reality Environments
- Immersive mixed reality environments offer a different approach to reproducing reality or embodied presence (Nakevska, 2012).
- The user is exposed to a multi- dimensional environment developed from a heterogeneous composition of technologies including sensors, augmented reality, augmented virtuality supported by processing applications and components that manage the use of contextual information without exclusion of the real-physical environment.
- Similar to CAVEs which focus on virtual worlds, the immersive mixed reality environment aims to create a ”fantasy” world where the user is engaged using multisensory augmentations of the surrounding environment.
- They may be expanded or moved along with the user thanks to the use of multiple geographically displaced markers.
- Museums may associate unique markers to a sequence of displayed exhibits spread out across several rooms and corridors that would provide an immersive mixed reality environment useful for providing more information.
2.3 Mixed reality and mobile devices
- The Augmented Reality (AR) form of mixed-reality is already present in many every-day applications, that are location or context aware, including the live- television broadcast of sports events (Azuma et. al, 2001) as it provides new ways of showing relationships and connections in the real world.
- (Uhomoibhi et. al., 2011) and (Andujar et. al., 2011) show the use of augmented reality in education and (FitzGerald, 2012) reported examples of AR applications from specific domains such as architecture and tourism, that engage the user in an exploratory role (like in games) aimed at the discovery of additional material or content.
- Mobile devices also contain one or more of the following sensors: microphone, multi-touch input , camera, location (gps), accelerometer (for acceleration, rotation or orientation), ambient light level, which may be used to aid the augmentation process.
- Many of the existing examples of mixed-reality on mobile platforms focus on using AR in providing passive information (text, audio and video overlays) to users based on input from sensors about physical location, movement and gestures.
- Other works such as (Onime, Uhomoibhi and Pietrosemoli, 2015) document the use of Augmented Virtuality (AV) on mobile devices for estimating power output of solar panels.
3.1 Mixed Reality Visualization Cubicles
- A mixed reality visualization cubicle may be created using a spatial arrangement of multiple markers.
- One or more AR markers are placed on each wall of the cubicle and each one provides a windowed view of the virtual environment.
- Figure 3 shows the AR visualization cubicle jointly developed by Santa’s Co (a software development company from Reggio Emilia, Italy), the Ulster University (UU) and the International Centre for Theoretical Physics (ICTP).
- The semi-immersive AR environment is composed of four large A3 markers, while three were positioned vertically, each on a separate wall (left, right and front from perspective of a user) to cover a 180◦ horizontal angle; the fourth was placed horizontal on the floor to cover a 90◦ vertical angle.
- That is, using this configuration, the cubicle may be used to provide a wide-angle seamless 180◦ view of the virtual world in the horizontal direction seamlessly combined with a 90◦ angle in the vertical direction.
3.2 Mixed-reality Augmentation Markers
- The marker is a type of place holder located within the environment that acts as an identifiable reference point for insertion of objects in the mixed reality world.
- In its basic format, the visualisation of a two dimensional QR code using a suitable application (QR reader) would cause the opening of a pre-determined Uniform Resource Locator (URL).
- Shapefiles definitions of represented objects are used to facilitate the 3D rendering by a suitable graphics library or engine that also provide the ability to scale them.
- In a technique used in mobile AR, the marker image is decomposed into unique set(s) of simple shapes and angles, which is then registered or encoded within the AR application as the marker (Onime, Uhomoibhi and Radicella, 2015).
- The markers employed for the mixed-reality cubicle discussed in this paper were computer generated abstract patterns composed of random polygons in greyscale colour.
3.3 Creating mobile Augmented Reality (mAR) software
- See-through augmented reality on mobile devices Figure 4 shows the technical flow-chart for the sequence of steps implemented in a typical mixed reality (AR) application software.
- Consider the 3-axis accelerometer device shown in Figure 5a, which is com- posed of elastic elements and a suspended mass.
4.1 Applications
- VR and mixed reality technologies are already widely applied in sectors ranging from medicine, entertainment, education and interactive guides, nature and earth science.
- Within the cubicle described in this paper, the spatial arrangement of markers provided a wide-angle (180◦ horizontal + 90◦ vertical) exploration that was used in an educational context for the interactive visualisation of geospatial data representing landforms.
- This form of application is equally useful for conducting interactive visits to cities or other remote sites/locations in a manner that allows a user to travel along streets and also experiencing the sights, sounds and smells.
- The latter would be possible with coordinated use of specialized sensors that release precaptured scents.
- Mixed-reality based tools are gaining grounds as a new class of ”Big Data” visualisation tools capable of providing interactive exploration for growing research outputs/data, large or big datasets resulting from simulations and physical experiments such as the LHC (CERN, Geneva) or Genome related sequencing (Onime and Uhomobhi, 2016).
4.2 Familiarity with mixed reality technology
- Anonymous feedback was obtained from 174 academicians (researchers and students).
- Demographically, participants were from 7 different countries al- though primarily from two institutions.
- The consenting adult volunteers, who participated without incentives, risks and disadvantages in the international study were informed of the purpose, confidentiality of the study and the intended use of the collected data.
- The collected data show less than 30% had used mixed reality environments which is lower than expected.
- It is possible that mixed reality cubicles as discussed in this paper would improve knowledge about both AR and VR technologies (Onime, Uhomoibhi and Wang, 2016).
4.3 Limitations
- The cubicle provides a fully immersive experience if used with suitable AR goggles or headmounted devices.
- Tablets and normal smart-phones alone provide a windowed semi-immersive view limited by their display/screen sizes.
- The mobile technology based viewing devices used within the cubicle may experience issues related to poor visibility in the presence of strong ambient light.
- Within the current version of the cubicle, complex gestures or movements involving twohands is not yet possible as users would hold the mobile device in one hand and can only perform gestures with the other hand.
- The standard dimensions of a typical cubicle does not allow for group visualizations or use.
Did you find this useful? Give us your feedback
Citations
25 citations
17 citations
15 citations
15 citations
7 citations
Cites background from "A study of developments and applica..."
...Although AR is a promising technology in the educational context, its educational use and research are in its infancy (Uhomoibhi et al., 2020)....
[...]
References
3 citations
Related Papers (5)
Frequently Asked Questions (2)
Q2. What future works have the authors mentioned in the paper "A study of developments and applications of mixed reality cubicles and their impact on learning" ?
Future work include the creation of an Experience Lab by the Artificial Intelligence and Ap- plications Research Group at Ulster University, that involve the deployment of several enhanced mixed-reality visualization cubicles at Ulster University and the ICTP. In line with the IVIS4BigData reference model, The mixed reality cubicle would need to be extended for multidisciplinary Computer Supported Group Work ( CSCW ) based on open standards over various infrastructure including local wireless or mesh networks, the internet and Clouds.