A survey on human-aware robot navigation

Pointing gestures are widely used in robot navigation approaches nowadays. However, most approaches only use pointing gestures, and these have two major limitations. Firstly, they need to recognize pointing gestures all the time, which leads to long processing time and significant system overheads. Secondly, the user's pointing direction may not be very accurate, so the robot may go to an undesired place. To relieve these limitations, we propose a voice-guided pointing robot navigation approach named VGPN, and implement its prototype on a wheeled robot, TurtleBot 2. VGPN recognizes a pointing gesture only if voice information is insufficient for navigation. VGPN also uses voice information as a supplementary channel to help determine the target position of the user's pointing gesture. In the evaluation, we compare VGPN to the pointing-only navigation approach. The results show that VGPN effectively reduces the processing time cost when pointing gesture is unnecessary, and improves the user satisfaction with navigation accuracy.

VGPN: Voice-Guided Pointing Robot Navigation for Humans

This paper implements the design for a robot that can be controlled simply by using interactive inputs from the operator such as voice and gesture along with object tracking. The system aims to create a prototype of a futuristic automated personal assistant for domestic as well as industrial purposes. Google text to speech API and Grassfire algorithm is used to control the basic locomotion of the system. The robot consists of a gripper arm which is used to pick and hold objects as desired by the operator.

Voice Assisted and Gesture Controlled Companion Robot

Numerous research have demonstrated that Convolutional Neural Network (CNN) models are capable of classifying visual field (VF) defects with great accuracy. In this study, we evaluated the performance of different pre-trained models (VGG-Net, MobileNet, ResNet, and DenseNet) in classifying VF defects and produced a comprehensive comparative analysis to compare the performance of different CNN models before and after hyperparameter tuning and fine-tuning. Using 32 batch sizes, 50 epochs, and ADAM as the optimizer to optimize weight, bias, and learning rate, VGG-16 obtained the highest accuracy of 97.63 percent, according to experimental findings. Subsequently, Bayesian optimization was utilized to execute automated hyperparameter tuning and automated fine-tuning layers of the pre-trained models to determine the optimal hyperparameter and fine-tuning layer for classifying many VF defect with the highest accuracy. We found that the combination of different hyperparameters and fine-tuning of the pre-trained models significantly impact the performance of deep learning models for this classification task. In addition, we also discovered that the automated selection of optimal hyperparameters and fine-tuning by Bayesian has significantly enhanced the performance of the pre-trained models. The results observed the best performance for the DenseNet-121 model with a validation accuracy of 98.46% and a test accuracy of 99.57% for the tested datasets.

/pdf/a-comprehensive-performance-analysis-of-transfer-learning-2vifz4fh.pdf

A Comprehensive Performance Analysis of Transfer Learning Optimization in Visual Field Defect Classification

A novel method for improving the accuracy of automatic speech recognition system by adding non-acoustic parameters are discussed in this paper. The gestural features which are commonly co-expressive with speech is considered for improving the accuracy of ASR system in noisy environment. Both dynamic and static gestures are integrated with speech recognition system and tested in various environmental conditions, i.e., noise levels. The accuracy of continuous speech recognition system and isolated word recognition system are tested with and without gestures under various noise conditions. The addition of visual features provides stable recognition accuracy under different environmental noise conditions for acoustic signals.

Enhanced Automatic Speech Recognition with Non-acoustic Parameters

Human-computer interaction is a hot topic in artificial intelligence. Artificial navigation is an interesting application of human-computer interaction, which control the action of the target device by speech or gestures information. The main virtue of artificial navigation is that it can control target device within a distance without any remote control device. This technology can be used in the areas of robot navigation, vehicle navigation in the industrial site and virtual reality. This paper proposed an algorithm for robot navigation which combining gesture recognition with speech recognition. Firstly, use nine gesture instructions and nine voice commands to establish reference models. Secondly, extract real-time Skeleton information and the current speech messages by Kinect. Thirdly, evaluate the fitness of the current gesture and speech information of the reference model. Finally, deduce the navigation control instructions to command the robot's movement. Gestures and speech can compensate for each other's deficiencies, improve the recognition rate and robustness of the algorithm, reduce the computational complexity of algorithm, makes the human-computer interaction more simple, clear and natural.

Ze Lei

Papers

Artificial robot navigation based on gesture and speech recognition