Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector

The ability to follow a human is an important requirement for a service robot designed to work along side humans in homes or in work places. This paper describes the development and implementation of a novel robust visual controller for the human-following robot. This visual controller consists of two parts: 1) a robust algorithm that tracks a human visible in its camera view and 2) a servo controller that generates necessary motion commands so that the robot can follow the target human. The tracking algorithm uses point-based features, like speeded up robust feature, to detect human under challenging conditions, such as, variation in illumination, pose change, full or partial occlusion, and abrupt camera motion. The novel contributions in the tracking algorithm include the following: 1) a dynamic object model that evolves over time to deal with short-term changes, while maintaining stability over long run; 2) an online K-D tree-based classifier along with a Kalman filter is used to differentiate a case of pose change from a case of partial or full occlusion; and 3) a method is proposed to detect pose change due to out-of-plane rotations , which is a difficult problem that leads to frequent tracking failures in a human following robot. An improved version of a visual servo controller is proposed that uses feedback linearization to overcome the chattering phenomenon present in sliding mode-based controllers used previously. The efficacy of the proposed approach is demonstrated through various simulations and real-life experiments with an actual mobile robot platform.

A Novel Vision-Based Tracking Algorithm for a Human-Following Mobile Robot

A wide range of human–robot collaborative applications in diverse domains, such as manufacturing, health care, the entertainment industry, and social interactions, require an autonomous robot to fo...

Person-following by autonomous robots: A categorical overview:

A method for using a remote vehicle having a stereo vision camera to detect, track, and follow a person, the method comprising: detecting a person using a video stream from the stereo vision camera and histogram of oriented gradient descriptors; estimating a distance from the remote vehicle to the person using depth data from the stereo vision camera; tracking a path of the person and estimating a heading of the person; and navigating the remote vehicle to an appropriate location relative to the person.

Person Following Using Histograms of Oriented Gradients

Due to its simplicity and efficiency, the pure-pursuit path tracking method has been widely employed for planned navigation of nonholonomic ground vehicles. In this paper, we investigate the application of this technique for reactive tracking of paths that are implicitly defined by perceived environmental features. Goal points are obtained through an efficient interpretation of range data from an onboard 2D laser scanner to follow persons, corridors, and walls. Moreover, this formulation allows that a robotic mission can be composed of a combination of different types of path segments. These techniques have been successfully tested in the tracked mobile robot Auriga-α in an indoor environment.

/pdf/pure-pursuit-reactive-path-tracking-for-nonholonomic-mobile-1zw1496kw0.pdf

Pure-pursuit reactive path tracking for nonholonomic mobile robots with a 2D laser scanner

We have developed the person following robot "ApriAttendatrade". This robot can accompany a person using vision based target detection and avoid obstacles with ultrasonic sensors while following the person. The robot first identifies an individual with its image processing system by detecting a person's region and recognizing the registered color and texture of his/her clothes. Usually, the person following robot has to detect and recognize the specified person and calculate his/her position in a complicated real-life environment of fixed objects and moving people. Our newly developed algorithm allows the robot to extract a particular individual from a cluttered background, and to find and reconnect with the person if it loses visual contact. Tracking people with stereo vision was realized by systematizing visual and motion control with a robust algorithm that utilizes various characteristics of the image data. The developed algorithm uses several analyses to extract information on the distance to each feature point, speed of target, color and texture of clothes for a stable tracking in many situations including changes of view due to self motion, shifts in lighting and objects similar to the target. The person following robot ApriAttendatrade has been exhibited at Aichi EXPO 2005, and its robust functions and smooth person following capability were successfully demonstrated

Development of a Person Following Robot with Vision Based Target Detection

Bin-picking of small parcels and other textureless planar-faced objects is a common task at warehouses. A general color image-based vision-guided robot picking system requires feature extraction and goal image preparation of various objects. However, feature extraction for goal image matching is difficult for textureless objects. Further, prior preparation of huge numbers of goal images is impractical at a warehouse. In this paper, we propose a novel depth image-based vision-guided robot bin-picking system for textureless planar-faced objects. Our method uses a deep convolutional neural network (DCNN) model that is trained on 15,000 annotated depth images synthetically generated in a physics simulator to directly predict grasp points without object segmentation. Unlike previous studies that predicted grasp points for a robot suction hand with only one vacuum cup, our DCNN also predicts optimal grasp patterns for a hand with two vacuum cups (left cup on, right cup on, or both cups on). Further, we propose a surface feature descriptor to extract surface features (center position and normal) and refine the predicted grasp point position, removing the need for texture features for vision-guided robot control and sim-to-real modification for DCNN model training. Experimental results demonstrate the efficiency of our system, namely that a robot with 7 degrees of freedom can pick randomly posed textureless boxes in a cluttered environment with a 97.5% success rate at speeds exceeding 1000 pieces per hour.

/pdf/depth-image-based-deep-learning-of-grasp-planning-for-3qu16j8ab0.pdf

Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking.

A robotic transportation system for shopping assistance has been developed. The system consists of a guidance robot, a cart robot, and environmental cameras. The guidance robot is an autonomous mobile robot with a position localization function, the cart robot with a shopping basket can follow the guidance robot, and environmental cameras can detect the robot and humans. A verification test has been performed in the test environment assumed for a food store. The guidance robot was able to guide the person to his/her desired place and to follow the person, and the cart robot was able to carry the articles while a shopping demonstration. Furthermore, safety technologies in the environment where robots and humans co-exist have been developed.

Development of robotic transportation system - Shopping support system collaborating with environmental cameras and mobile robots -

Self-localization is important for mobile robots in order to move accurately, and many works use an omnidirectional camera for self-localization. However, it is difficult to realize fast and accurate self-localization by using only one omnidirectional camera without any calibration. For its realization, we use “tracked scale and rotation invariant feature points” that are regarded as landmarks. These landmarks can be tracked and do not change for a “long” time. In a landmark selection phase, robots detect the feature points by using both a fast tracking method and a slow “Speed Up Robust Features (SURF)” method. After detection, robots select landmarks from among detected feature points by using Support Vector Machine (SVM) trained by feature vectors based on observation positions. In a self-localization phase, robots detect landmarks while switching detection methods dynamically based on a tracking error criterion that is calculated easily even in the uncalibrated omnidirectional image. We performed experiments in an approximately 10 [m] × 10 [m] mock supermarket by using a navigation robot ApriTau™ that had an omnidirectional camera on its top. The results showed that ApriTau™ could localize 2.9 times faster and 4.2 times more accurately by using the developed method than by using only the SURF method. The results also showed that ApriTau™ could arrive at a goal within a 3 [cm] error from various initial positions at the mock supermarket.

Mobile robot self-localization based on tracked scale and rotation invariant feature points by using an omnidirectional camera

Online planning agents can adapt to the dynamic environment by continuously modifying plans during the plan execution. This adaptability of the online planning agents is based on the assumption that they can recognise the events happening in the environment. However, robotic agents working in the real environment cannot obtain necessary information from the outer world by simply turning all sensors on while executing a plan. They have to actively sense the world by changing the direction of the sensor to the target, analysing the raw sensor data, extracting important information and recognising the situation. Online planning is useless if they cannot obtain necessary information from the world. Therefore, it is crucial to decide what events the agent needs to recognise in the background while executing a plan. To solve this problem, this paper introduces a new background sensing control technique by which planning agents can effectively observe the real environment and obtain important information when necessary during the plan execution.

Seiji Tokura

Papers

Development of a Person Following Robot with Vision Based Target Detection

Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking.

Development of robotic transportation system - Shopping support system collaborating with environmental cameras and mobile robots -

Mobile robot self-localization based on tracked scale and rotation invariant feature points by using an omnidirectional camera

Background sensing control for planning agents working in the real world