The aim of this article is focused on the design of an obstacle detection system for assisting visually impaired people. A dense disparity map is computed from the images of a stereo camera carried by the user. By using the dense disparity map, potential obstacles can be detected in 3D in indoor and outdoor scenarios. A ground plane estimation algorithm based on RANSAC plus filtering techniques allows the robust detection of the ground in every frame. A polar grid representation is proposed to account for the potential obstacles in the scene. The design is completed with acoustic feedback to assist visually impaired users while approaching obstacles. Beep sounds with different frequencies and repetitions inform the user about the presence of obstacles. Audio bone conducting technology is employed to play these sounds without interrupting the visually impaired user from hearing other important sounds from its local environment. A user study participated by four visually impaired volunteers supports the proposed system.

https://www.mdpi.com/1424-8220/12/12/17476/pdf

Assisting the Visually Impaired: Obstacle Detection and Warning System by Acoustic Feedback

In this paper we introduce a real-time obstacle detection and classification system designed to assist visually impaired people to navigate safely, in indoor and outdoor environments, by handling a smartphone device. We start by selecting a set of interest points extracted from an image grid and tracked using the multiscale Lucas - Kanade algorithm. Then, we estimate the camera and background motion through a set of homographic transforms. Other types of movements are identified using an agglomerative clustering technique. Obstacles are marked as urgent or normal based on their distance to the subject and the associated motion vector orientation. Following, the detected obstacles are fed/sent to an object classifier. We incorporate HOG descriptor into the Bag of Visual Words (BoVW) retrieval framework and demonstrate how this combination may be used for obstacle classification in video streams. The experimental results demonstrate that our approach is effective in image sequences with significant camera motion and achieves high accuracy rates, while being computational efficient.

/pdf/a-smartphone-based-obstacle-detection-and-classification-17aeh1w1rz.pdf

A Smartphone-Based Obstacle Detection and Classification System for Assisting Visually Impaired People

Any mobility aid for the visually impaired people should be able to accurately detect and warn about nearly obstacles. In this paper, we present a method for support system to detect obstacle in indoor environment based on Kinect sensor and 3D-image processing. Color-Depth data of the scene in front of the user is collected using the Kinect with the support of the standard framework for 3D sensing OpenNI and processed by PCL library to extract accurate 3D information of the obstacles. The experiments have been performed with the dataset in multiple indoor scenarios and in different lighting conditions. Results showed that our system is able to accurately detect the four types of obstacle: walls, doors, stairs, and a residual class that covers loose obstacles on the floor. Precisely, walls and loose obstacles on the floor are detected in practically all cases, whereas doors are detected in 90.69% out of 43 positive image samples. For the step detection, we have correctly detected the upstairs in 97.33% out of 75 positive images while the correct rate of downstairs detection is lower with 89.47% from 38 positive images. Our method further allows the computation of the distance between the user and the obstacles.

Real-Time Obstacle Detection System in Indoor Environment for the Visually Impaired Using Microsoft Kinect Sensor

This paper presents a computer vision based sensory substitution device for the visually impaired. Its main objective is to provide the users with a 3D representation of the environment around them, conveyed by means of the hearing and tactile senses. One of the biggest challenges for this system is to ensure pervasiveness, i.e., to be usable in any indoor or outdoor environments and in any illumination conditions. This work reveals both the hardware (3D acquisition system) and software (3D processing pipeline) used for developing this sensory substitution device and provides insight on its exploitation in various scenarios. Preliminary experiments with blind users revealed good usability results and provided valuable feedback for system improvement.

/pdf/computer-vision-for-the-visually-impaired-the-sound-of-3u8gxsqyar.pdf

Computer Vision for the Visually Impaired: the Sound of Vision System

This study proposes an obstacle detection method that uses depth information to allow the visually impaired to avoid obstacles when they move in an unfamiliar environment. The system is composed of three parts: scene detection, obstacle detection and a vocal announcement. This study proposes a new method to remove the ground plane that overcomes the over-segmentation problem. This system addresses the over-segmentation problem by removing the edge and the initial seed position problem for the region growth method using the Connected Component Method (CCM). This system can detect static and dynamic obstacles. The system is simple, robust and efficient. The experimental results show that the proposed system is both robust and convenient.

An Indoor Obstacle Detection System Using Depth Information and Region Growth.

An obstacle system based on visual selective feature and stereo vision is stated in this paper. By extracting feature vectors with color, intensity and direction, the corresponding saliency map is constructed. Then, obstacle area is figured out through threshold segmentation to the saliency map, and 3-D information of the obstacles is computed by using stereo vision. Finally, obstacle information is transformed into voice which can be delivered to the blind. Indoor and outdoor experiment has proved practicality and effectiveness of this method.

Obstacle Detection System for Visually Impaired People Based on Stereo Vision

Target posture measuring method based on binocular vision under double mediums

The final approach phase of spacecraft rendezvous and docking is extremely important. In order to solve the problem of the real-time acquisition of the relative pose between target and spacecraft in near distance (<2m), this paper established a binocular stereovision model, and proposed a non-cooperative target relative pose measuring method based on stereo vision and RANSAC algorithm. Linear characteristic of Non-cooperative target was used to abstract feature points firstly, then stereo matching and 3-D restructuring were taken for the feature points, finally, an algorithm based on RANSAC algorithm was used to calculate the relative pose between the target and the camera. Therefore, errors were eliminated effectively, and the computation load was decreased by using disparity gradient constraint. Experimental results show that high accuracy and real-time results are the advantages of this method.

/pdf/relative-pose-measurement-algorithm-of-non-cooperative-5edoxt696o.pdf

Relative Pose Measurement Algorithm of Non-cooperative Target based on Stereo Vision and RANSAC

In order to solve the problem of the non-cooperative target relative pose measurement in near distance (

Binocular Vision-based Position Determination Algorithm and System

The invention discloses an information acquisition and transfer method for assistant vision systems. The method comprises the following steps: (1) extracting two original digital images of an object in different angles by using two cameras at the same time; (2) extracting characteristic points of the two original digital images by means of the Harris corner detection; (3) extracting three-dimensional geometrical information of the characteristic points by using the two cameras; (4) making a rectangular region where each characteristic point serves as the center, finding out the position of the next frame characteristic point and calculating motion vectors of the characteristic point; (5) dividing the road surface information of the original digital image by using a color histogram, according to the chromatic information and calculating the road information; (6) coding the motion information of the characteristic point of the original image, the three-dimensional geometrical information of the characteristic point and the road information respectively; and (7) transferring the coded information to a person with vision disorders via the information transfer array unit in the assistant vision system. The information acquisition and transfer method is advantageous in the accurate extraction of three-dimensional geometrical information of the object, and helps the patients with vision disorders to walk directionally and safely.

Long Chen

Papers

Obstacle Detection System for Visually Impaired People Based on Stereo Vision

Target posture measuring method based on binocular vision under double mediums

Relative Pose Measurement Algorithm of Non-cooperative Target based on Stereo Vision and RANSAC

Binocular Vision-based Position Determination Algorithm and System

Information acquisition and transfer method of auxiliary vision system