A Comparison of Virtual and Physical Training Transfer of Bimanual Assembly Tasks
TL;DR: A study that compares the effectiveness of virtual training and physical training for teaching a bimanual assembly task and highlights the validity of virtual reality systems in training.
Abstract: As we explore the use of consumer virtual reality technology for training applications, there is a need to evaluate its validity compared to more traditional training formats. In this paper, we present a study that compares the effectiveness of virtual training and physical training for teaching a bimanual assembly task. In a between-subjects experiment, 60 participants were trained to solve three 3D burr puzzles in one of six conditions comprised of virtual and physical training elements. In the four physical conditions, training was delivered via paper- and video-based instructions, with or without the physical puzzles to practice with. In the two virtual conditions, participants learnt to assemble the puzzles in an interactive virtual environment, with or without 3D animations showing the assembly process. After training, we conducted immediate tests in which participants were asked to solve a physical version of the puzzles. We measured performance through success rates and assembly completion testing times. We also measured training times as well as subjective ratings on several aspects of the experience. Our results show that the performance of virtually trained participants was promising. A statistically significant difference was not found between virtual training with animated instructions and the best performing physical condition (in which physical blocks were available during training) for the last and most complex puzzle in terms of success rates and testing times. Performance in retention tests two weeks after training was generally not as good as expected for all experimental conditions. We discuss the implications of the results and highlight the validity of virtual reality systems in training.
Summary (3 min read)
- Section 3 presents the experimental design and hypotheses.
- Section 6 discusses the results, limitations and future work.
- A virtual replica of the laboratory was modeled for the virtual enviornment used in the virtual experimental conditions.
- An Oculus Rift Consumer Version 1, two Oculus Touch controllers and two Oculus sensors were used for the virtual experimental conditions.
- Preassembled blocks for the first and second puzzles were glued together.
4.3 Physical training environment
- For those experimental conditions in which the physical blocks were available during training (PB and PV I B) these were initially placed on the table following the same configuration as the paper instructions.
- Preassembled puzzles were placed behind the blocks.
4.4 Virtual training environment
- All interactions in the virtual training environment could be equally carried out using either hand and participants could concurrently complete one interaction with each hand.
- A participant could grab and rotate the assembled pieces with one hand and grab the next block to attach with the other hand.
- The green highlight indicates on the block is colliding with its preview block and within twenty degrees from the correct orientation.
- By releasing the trigger button of the Oculus Touch controller the virtual block would snap into its correct location.
- After a waiting period of two weeks, participants returned to the lab for the second session.
- In this session participants were asked to complete a paper version of the Vandenberg and Kuse Mental Rotations Test  .
- They then completed the retention test for each of the three puzzles, in which they were asked to solve the three burr puzzles from the first session without a training phase, in the same order and in a maximum of three minutes.
- They completed the same questionnaire from the first session at the end of each retention trial (see Table 3 ).
- After completing all retention trials they were interviewed regarding strategies used throughout the session.
5.1 Types of errors
- Unsuccessful puzzle completions during immediate and retention testing were due to one of two reasons.
- In most cases, participants did not complete the 3D puzzles within the given maximum time (180s).
- On the other hand, a low number of participants decided to stop the time before the upper limit thinking that they had successfully solved the puzzle.
- Close inspection showed that they had not correctly assembled the pieces.
- Completion time values for both immediate and retention testing were corrected by assigning the upper time limit (180s) to all unsuccessful attempts.
5.2 First session 5.2.1 Training times
- The post hoc analysis revealed statistically significant differences in training times for the first puzzle.
- There was a statistically significant difference between P (mean rank = 15.
- The post hoc analysis revealed statistically significant differences in training times for the second puzzle.
5.2.2 Immediate testing success rates
- The model suggested that participants in the P experimental condition were 0.074 times as likely to successfully assemble the third puzzle than participants in the reference category (PV I B).
- The model suggested that participants in the PV I experimental condition were 0.028 times as likely to successfully assemble the third puzzle than participants in the reference category (PV I B).
- Likert scale extremes Difficulty Please rate the difficulty of the task you just completed.
- It is important to note that all participants in this condition successfully completed the third puzzle.
- The model suggested that participants who succeeded at correctly assembling the second puzzle were 9.687 times as likely to successfully assemble the third puzzle than participants in the reference category (PV I B).
- For the third puzzle, the binomial logistic regression model with the highest percentage of correctly classified observations was the one that ascertained the effect of both experimental condition and successful completion of the previous puzzle.
- Test statistics using Dunn's procedure  for immediate testing times between the different experimental conditions.
5.2.3 Immediate testing completion times
- The post hoc analysis revealed statistically significant differences in immediate testing times for the third puzzle.
- The analysis of immediate testing completion times shows some support for H2 and H3.
5.2.4 Subjective questionnaire ratings
- There was a statistically significant difference in ease of use of the training environment (F(5,54) = 3.044, p = 0.017) between groups as determined by one-way ANOVA for the third puzzle.
- No other significant interactions were found for the third puzzle.
- A total of 56 participants that completed the first part session returned to complete the second session two weeks later (average number of days between training session and retention session: 14.16, SD = 0.918).
- Overall, retention testing performance was lower than expected for all conditions both in terms of success rates and completion times.
5.3.4 Subjective questionnaire ratings
- There was no statistically significant difference in rated difficulty and seriousness between groups as determined by one-way ANOVA for any of the three puzzles.
- Tukey post hoc tests showed no significant interactions.
- One of the limitations in their design was the high complexity of the puzzles.
- Overall, retention testing resulted in lower performance than the authors had expected and they believe this is due to the difficulty associated with remembering the process to solve the three puzzles two weeks after the training.
- This was further validated by verbal feedback from their participants during the second session.
- The authors previous piloting of the task had not shown this effect.
- Future studies should further evaluate the suitability of the task for retention.
- The authors analysed performance in terms of success rates as well as immediate testing times and retention testing times.
- The authors results show that the performance of virtually trained participants was promising.
- A statistically significant difference wasn't found between condition V E A and the best performing physical condition (PV I B, in which physical blocks and animated instructions were available during training) for the last and most complex puzzle in terms of success rates and immediate testing times.
- Retention testing performance was unexpectedly low due to the high complexity of the task.
- The authors believe that the results of this study further validate the effectiveness of virtual training for bimanual assembly tasks.
Did you find this useful? Give us your feedback
Cites background from "A Comparison of Virtual and Physica..."
...2016; Murcia-Lopez and Steed 2018; Werrlich et al....
...Also using burr puzzles, Murcia-Lopez and Steed (2018) sought to understand the effects of XR training when haptic devices or physical objects are not available during the training process....
Cites background or methods or result from "A Comparison of Virtual and Physica..."
...Given the background described above and based on previous research (Grealy et al., 1999; Lozano-Quilis et al., 2013; Murcia-López & Steed, 2018; Sampson et al., 2016), the following hypothesis was formulated: we expected that, compared to VRT and CPT, combining these two methods would have a more…...
...…training are the activation of pathways, the increase in the number of synapses, and the increase in the sensory area that is seen in plasticization (Ashton-Miller, Wojtys, Huston, & Fry-Welch, 2001; Baram & Miller, 2006; Grealy et al., 1999; Lozano-Quilis et al., 2013; Murcia-López & Steed, 2018)....
...Psychological problems have been reported for women with MS in addition to their motor and sensory problems (Lozano-Quilis et al., 2013; Murcia-López & Steed, 2018)....
...Moreover, a facilitative sense of control and confidence can be observed following VRT (Massetti et al., 2016; Murcia-López & Steed, 2018)....
...These results are consistent with those reported in previous studies (Grealy et al., 1999; LozanoQuilis et al., 2013; Massetti et al., 2016; Murcia-López & Steed, 2018; Sampson et al., 2016; Song et al., 2010)....
Cites background from "A Comparison of Virtual and Physica..."
...Understanding how to wield AR/VR tools to reshape how we perceive the world also has incredible potential for societal and clinical applications: in particular, for training in industrial tasks [6, 12, 28], in medical and surgical applications ,or for rehabilitation protocols [17, 32]....
"A Comparison of Virtual and Physica..." refers background in this paper
...Previous research has highlighted the effectiveness of immersive mixed reality training in different disciplines, including military training, medical training and vehicle driving simulators [17,21], as well as navigation and spatial knowledge training [8, 23], amongst others....