Showing papers on "Monocular vision published in 2015"

Automatic re-initialization and failure recovery for aggressive flight with a monocular vision-based quadrotor

Abstract: Autonomous, vision-based quadrotor flight is widely regarded as a challenging perception and control problem since the accuracy of a flight maneuver is strongly influenced by the quality of the on-board state estimate. In addition, any vision-based state estimator can fail due to the lack of visual information in the scene or due to the loss of feature tracking after an aggressive maneuver. When this happens, the robot should automatically re-initialize the state estimate to maintain its autonomy and, thus, guarantee the safety for itself and the environment. In this paper, we present a system that enables a monocular-vision-based quadrotor to automatically recover from any unknown, initial attitude with significant velocity, such as after loss of visual tracking due to an aggressive maneuver. The recovery procedure consists of multiple stages, in which the quadrotor, first, stabilizes its attitude and altitude, then, re-initializes its visual state-estimation pipeline before stabilizing fully autonomously. To experimentally demonstrate the performance of our system, we aggressively throw the quadrotor in the air by hand and have it recover and stabilize all by itself. We chose this example as it simulates conditions similar to failure recovery during aggressive flight. Our system was able to recover successfully in several hundred throws in both indoor and outdoor environments.

Design and Implementation of Autonomous Car using Raspberry Pi

Abstract: The project aims to build a monocular vision autonomous car prototype using Raspberry Pi as a processing chip. An HD camera along with an ultrasonic sensor is used to provide necessary data from the real world to the car. The car is capable of reaching the given destination safely and intelligently thus avoiding the risk of human errors. Many existing algorithms like lane detection, obstacle detection are combined together to provide the necessary control to the car.

3D Perception Based Quality Pooling: Stereopsis, Binocular Rivalry, and Binocular Suppression

Abstract: One of the most challenging ongoing issues in the field of 3D visual research is how to interpret human 3D perception over virtual 3D space between the human eye and a 3D display. When a human being perceives a 3D structure, the brain classifies the scene into the binocular or monocular vision region depending on the availability of binocular depth perception in the unit of a certain region (coarse 3D perception). The details of the scene are then perceived by applying visual sensitivity to the classified 3D structure (fine 3D perception) with reference to the fixation. Furthermore, we include the coarse and fine 3D perception in the quality assessment, and propose a human 3D Perception-based Stereo image quality pooling (3DPS) model. In 3DPS we divide the stereo image into segment units, and classify each segment as either the binocular or monocular vision region. We assess the stereo image according to the classification by applying different visual weights to the pooling method to achieve more accurate quality assessment. In particular, it is demonstrated that 3DPS performs remarkably for quality assessment of stereo images distorted by coding and transmission errors.

A Self-learning Framework for Statistical Ground Classification using Radar and Monocular Vision

Abstract: Reliable terrain analysis is a key requirement for a mobile robot to operate safely in challenging environments, such as in natural outdoor settings. In these contexts, conventional navigation systems that assume a priori knowledge of the terrain geometric properties, appearance properties, or both, would most likely fail, due to the high variability of the terrain characteristics and environmental conditions. In this paper, a self-learning framework for ground detection and classification is introduced, where the terrain model is automatically initialized at the beginning of the vehicle's operation and progressively updated online. The proposed approach is of general applicability for a robot's perception purposes, and it can be implemented using a single sensor or combining different sensor modalities. In the context of this paper, two ground classification modules are presented: one based on radar data, and one based on monocular vision and supervised by the radar classifier. Both of them rely on online learning strategies to build a statistical feature-based model of the ground, and both implement a Mahalanobis distance classification approach for ground segmentation in their respective fields of view. In detail, the radar classifier analyzes radar observations to obtain an estimate of the ground surface location based on a set of radar features. The output of the radar classifier serves as well to provide training labels to the visual classification module. Once trained, the vision-based classifier is able to discriminate between ground and nonground regions in the entire field of view of the camera. It can also detect multiple terrain components within the broad ground class. Experimental results, obtained with an unmanned ground vehicle operating in a rural environment, are presented to validate the system. It is shown that the proposed approach is effective in detecting drivable surface, reaching an average classification accuracy of about 80% on the entire video frame with the additional advantage of not requiring human intervention for training or a priori assumption on the ground appearance.

Online self-supervised monocular visual odometry for ground vehicles

Abstract: This paper presents an online self-supervised approach to monocular visual odometry and ground classification applied to ground vehicles. We solve the motion and structure problem based on a constrained kinematic model. The true scale of the monocular scene is recovered by estimating the ground surface. We consider a general parametric ground surface model and use the Random Sample Consensus (RANSAC) algorithm for robust fitting of the parameters. The estimated ground surface provides training samples to learn a probabilistic appearance-based ground classifier in an online and self-supervised manner. The appearance-based classifier is then used to bias the RANSAC sampling to generate better hypotheses for parameter estimation of the ground surface model. Thus, without relying on any prior information, we combine geometric estimates with appearance-based classification to achieve an online self-learning scheme from monocular vision. Experimental results demonstrate that online learning improves the computational efficiency and accuracy compared to standard sampling in RANSAC. Evaluations on the KITTI benchmark dataset demonstrate the stability and accuracy of our overall methods in comparison to previous approaches.

A Lack of Experience-Dependent Plasticity After More Than a Decade of Recovered Sight

Abstract: In 2000, monocular vision was restored to M. M., who had been blind between the ages of 3 and 46 years. Tests carried out over 2 years following the surgery revealed impairments of 3-D form, object, and face processing and an absence of object- and face-selective blood-oxygen-level-dependent responses in ventral visual cortex. In the present research, we reexamined M. M. to test for experience-dependent recovery of visual function. Behaviorally, M. M. remains impaired in 3-D form, object, and face processing. Accordingly, we found little to no evidence of the category-selective organization within ventral visual cortex typically associated with face, body, scene, or object processing. We did observe remarkably normal object selectivity within lateral occipital cortex, consistent with M. M.’s previously reported shape-discrimination performance. Together, these findings provide little evidence for recovery of high-level visual function after more than a decade of visual experience in adulthood.

Vehicle ranging method based on monocular vision

Abstract: The invention discloses a vehicle ranging method based on monocular vision, and belongs to the field of target detection and ranging. The method comprises the steps of installing a monocular camera on a vehicle, measuring the height and the pitching angle of the camera, and determining focal distance parameters of the camera; acquiring a video image in an expressway environment; performing preliminary de-noising and filtration on the video image by adopting Gaussian filtration; performing interest region segmentation preprocessing before detection of a target vehicle on the video image; performing vehicle detection in a segmented video image region, wherein an Haar feature for increasing a wheel feature and a tail feature is adopted in the target vehicle detection, so as to effectively improve the accuracy of target vehicle identification; and measuring the distance of the target vehicle, wherein a ranging method based on pin-hole imaging is adopted within a short-distance range, while a ranging method of data fitting is adopted within a long-distance range, so that the error rate is reduced, and a real-time ranging effect can be achieved. The method has the advantages of high detection speed, high accuracy, strong real-time property and low cost.

Multiframe-Based High Dynamic Range Monocular Vision System for Advanced Driver Assistance Systems

Abstract: In this paper, we propose a monocular multiframe high dynamic range (HDR) monocular vision system to improve the imaging quality of traditional CMOS/charge-coupled device (CCD)-based vision system for advanced driver assistance systems (ADASs). Conventional CMOS/CCD image sensors are confined to limited dynamic range that it impairs the imaging quality under undesirable environments for ADAS (e.g., strong contrast of bright and darkness, strong sunlight, headlights at night, and so on). Contrary to current HDR video solutions relying on expensive specially designed sensors, we implement a multiframe HDR algorithm to enable one common CMOS/CCD sensor capturing HDR video. Key parts of the realized HDR vision system are: 1) circular exposure control; 2) latent image calculation; and 3) exposure fusion. We have successfully realized a prototype of monocular HDR vision system and mounted it on our SetCar platform. The effectiveness of this technique is demonstrated by our experimental results, while its bottleneck is the processing time. By exploring the capability of the proposed method in the future, a low-cost HDR vision system can be achieved for ADAS.

Practical rotation angle measurement method by monocular vision

Abstract: A practical vision-based method is proposed to measure single axis rotation angles. Compared with existing angle measurement methods, the proposed method is more practical because of the simple equipment required and an easy installation. Furthermore, unlike most existing methods, the proposed method does not require the rotation axis information at all. The information is calibrated by two-view geometry of the single axis motion. Then, on the basis of the calibration results, an angle estimation algorithm with point matching is proposed. Experiments on synthetic and real images show that the proposed method is robust with respect to image noises and occlusion. A measurement accuracy of less than 0.1° is obtained in real experiments only using a provided camera and a normal printed checkboard. Moreover, a single axis rotation angle measurement MATLAB toolbox is developed, which is available online (http://quanquan.buaa.edu.cn/Anglemeasurementtoolbox.html).

Real-time obstacle detection with motion features using monocular vision

Abstract: Real-time obstacle detection by monocular vision is a challenging problem in autonomous navigation of vehicles and driver-assistance systems. In this paper, we introduce an approach of real-time obstacle detection which can effectively tell apart obstacles from shadows and road surface markings. We propose the followings: (1) a two consecutive frames (TCF) model to find the differences between obstacles and the ground plane by motion features; (2) a filter to increase probabilities of obstacle regions; (3) an updating process to reduce false positives and update the algorithm when the vehicle moves on. We perform experiments on two datasets and our autonomous vehicle. The results show that our method is effective in various conditions and meets the real-time requirement.

Pose estimation using monocular vision and inertial sensors aided with ultra wide band

Abstract: This paper presents a method for global pose estimation using inertial sensors, monocular vision, and ultra wide band (UWB) sensors. It is demonstrated that the complementary characteristics of these sensors can be exploited to provide improved global pose estimates, without requiring the introduction of any visible infrastructure, such as fiducial markers. Instead, natural landmarks are jointly estimated with the pose of the platform using a simultaneous localization and mapping framework, supported by a small number of easy-to-hide UWB beacons with known positions. The method is evaluated with data from a controlled indoor experiment with high precision ground truth. The results show the benefit of the suggested sensor combination and suggest directions for further work.

Object posture measuring method based on CAD model and monocular vision

Abstract: The invention discloses an object posture measuring method based on a CAD model and monocular vision. The object posture measuring method based on the CAD model and the monocular vision comprises the following steps: after obtaining a movement relationship between a template shooting camera of an assumed movement and a fixed camera by virtue of movement assumption and iterative calculation of the template shooting camera to obtain an outer parameter of a binocular system consisting of the template shooting camera of the assumed movement and the fixed camera, performing three-dimensional reconstruction on a target to obtain three-dimensional point cloud data of the target, and rectifying a CAD model including object three-dimensional structure information to obtain a corresponding relation between the object posture of the target object under a current world coordinate system and the CAD model, and accurately figuring the posture of the target object. A movable camera fixing carrier is not needed and the CAD model and the three-dimensional vision information of the object are combined, so that the cost is low, the precision is high, and the demand of the actual industrial application can be met.

Toward 3D reconstruction of outdoor scenes using an MMW radar and a monocular vision sensor.

Abstract: In this paper, we introduce a geometric method for 3D reconstruction of the exterior environment using a panoramic microwave radar and a camera. We rely on the complementarity of these two sensors considering the robustness to the environmental conditions and depth detection ability of the radar, on the one hand, and the high spatial resolution of a vision sensor, on the other. Firstly, geometric modeling of each sensor and of the entire system is presented. Secondly, we address the global calibration problem, which consists of finding the exact transformation between the sensors’ coordinate systems. Two implementation methods are proposed and compared, based on the optimization of a non-linear criterion obtained from a set of radar-to-image target correspondences. Unlike existing methods, no special configuration of the 3D points is required for calibration. This makes the methods flexible and easy to use by a non-expert operator. Finally, we present a very simple, yet robust 3D reconstruction method based on the sensors’ geometry. This method enables one to reconstruct observed features in 3D using one acquisition (static sensor), which is not always met in the state of the art for outdoor scene reconstruction. The proposed methods have been validated with synthetic and real data.

Passive target tracking of marine traffic ships using onboard monocular camera for unmanned surface vessel

Abstract: Enhancing the performance of passive target tracking and trajectory estimation of marine traffic ships is focused using a monocular camera mounted on an unmanned surface vessel. To accurately estimate the trajectory of a target traffic ship, the relative bearing and range information between the observing ship and the target ship is required. Monocular vision provides bearing information with reasonable accuracy but with no explicit range information. The relative range information can be extracted from the bearing changes induced by the relative ship motion in the framework of bearings-only tracking (BOT). BOT can be effective in crossing situations with large bearing angle changes. However, it often fails in head-on or overtaking situations due to small bearing angle changes and the resulting low observability of the tracking filter. To deal with the lack of observability, the vertical pixel distance between the horizon and the target ship in the image is used, which improves the overall target tracking performance. The feasibility and performance of the proposed tracking approach were validated through field experiments at sea.

Experimental vision regulation of a quadrotor

Abstract: In this paper, the vision regulation of a quadrotor is introduced. The objective of the regulation is to maintain the vehicle in a desired and constant position. The proposed strategy consists in three steps: 1) a camera is employed as a retro to obtain the position, 2) the optical flow of the Lucas and Kanade method is applied to obtain the linear velocity, 3) the position and velocity are employed by a proportional integral derivative control (PID) with nested saturation for the quadrotor regulation. In addition, the comparison between the monocular and stereo visions is showed.

Eye movements in chameleons are not truly independent – evidence from simultaneous monocular tracking of two targets

Abstract: Chameleons perform large-amplitude eye movements that are frequently referred to as independent, or disconjugate. When prey (an insect) is detected, the chameleon's eyes converge to view it binocularly and 'lock' in their sockets so that subsequent visual tracking is by head movements. However, the extent of the eyes' independence is unclear. For example, can a chameleon visually track two small targets simultaneously and monocularly, i.e. one with each eye? This is of special interest because eye movements in ectotherms and birds are frequently independent, with optic nerves that are fully decussated and intertectal connections that are not as developed as in mammals. Here, we demonstrate that chameleons presented with two small targets moving in opposite directions can perform simultaneous, smooth, monocular, visual tracking. To our knowledge, this is the first demonstration of such a capacity. The fine patterns of the eye movements in monocular tracking were composed of alternating, longer, 'smooth' phases and abrupt 'step' events, similar to smooth pursuits and saccades. Monocular tracking differed significantly from binocular tracking with respect to both 'smooth' phases and 'step' events. We suggest that in chameleons, eye movements are not simply 'independent'. Rather, at the gross level, eye movements are (i) disconjugate during scanning, (ii) conjugate during binocular tracking and (iii) disconjugate, but coordinated, during monocular tracking. At the fine level, eye movements are disconjugate in all cases. These results support the view that in vertebrates, basic monocular control is under a higher level of regulation that dictates the eyes' level of coordination according to context.

A monocular vision-aided inertial navigation system with improved numerical stability

Abstract: Presented at the AIAA Guidance Navigation and Control Conference, Kissimmee, Florida, January 2015.

Monocular vision-based autonomous navigation system on a toy quadcopter in unknown environments

Abstract: In this paper, we present an monocular vision-based autonomous navigation system for a commercial quadcoptor. The quadcoptor communicates with a ground-based laptop via wireless connection. The video stream of the front camera on the drone and the navigation data measured on-board are sent to the ground station and then processed by a vision-based SLAM system. In order to handle motion blur and frame lost in the received video, our SLAM system consists of a improved robust feature tracking scheme and a relocalisation module which achieves fast recovery from tracking failure. An Extended Kalman filter (EKF) is designed for sensor fusion. Thanks to the proposed EKF, accurate 3D positions and velocities can be estimated as well as the scaling factor of the monocular SLAM. Using a motion capture system with millimeter-level precision, we also identify the system models of the quadcoptor and design the PID controller accordingly. We demonstrate that the quadcoptor can navigate along pre-defined paths in an unknown indoor environment with our system using its front camera and onboard sensors only after some simple manual initialization procedures.

New Depth From Focus Filters in Active Monocular Vision Systems for Indoor 3-D Tracking

Abstract: In this paper, new methodologies for the estimation of the depth of a target with unknown dimensions, based on depth from focus strategies, are proposed. The measurements are extracted from images acquired with a single camera, resorting to the minimization of a new functional, deeply rooted on the optical characteristics of the lens system. The analysis and synthesis of two complementary filters and a linear parametrically varying observer are discussed in detail. These estimators use information present on the boundary of the target, which is assumed to be on a plane parallel to the camera sensor, and whose dimensions are considered to remain constant over time. This paper complements a single pan and tilt camera-based indoor positioning and tracking system. To assess the performance of the proposed solutions, a series of indoor experimental tests for a range of operation of up to ten meters, which included tracking and localizing a small unmanned aerial vehicle with unknown dimensions, was carried out. Depth estimates with accuracies on the order of a few centimeters were obtained.

Visual road following using intrinsic images

Abstract: We present a real-time visual-based road following method for mobile robots in outdoor environments. The approach combines an image processing method, that allows to retrieve illumination invariant images, with an efficient path following algorithm. The method allows a mobile robot to autonomously navigate along pathways of different types in adverse lighting conditions using monocular vision. To validate the proposed method, we have evaluated its ability to correctly determine boundaries of pathways in a challenging outdoor dataset. Moreover, the method's performance was tested on a mobile robotic platform that autonomously navigated long paths in urban parks. The experiments demonstrated that the mobile robot was able to identify outdoor pathways of different types and navigate through them despite the presence of shadows that significantly influenced the paths' appearance.

Point character based monocular vision pose measurement method

Abstract: The invention discloses a point character based monocular vision pose measurement method oriented to the practical engineering application by adopting a reasonably designed measurement system via fully utilizing the technical means of model constraint, redundancy character point mutual testing and the like, which can be used for measuring the relative positions and relative poses of two objects, and can be applied to complex variable background and illumination conditions. The point character based monocular vision pose measurement method comprises the following steps: firstly, a camera acts as a measurement sensor, and is matched with a special point character cooperative marker to form a measurement system; secondly, the camera is utilized for photographing an image comprising the cooperative marker, and a character point image region combination which meets the marker model constraint is found out via image processing to act as a candidate marker; thirdly, the relative positions and relative poses of two objects are calculated by utilizing a P3P measurement method; fourthly, the final test result is obtained by adopting a redundancy information mutual testing method. According to the invention, the problem of multi-solution, the problem of speed calculation and the problem of stability of a three-point character monocular vision measurement are solved, and the measurement precision is improved.

A variational approach to online road and path segmentation with monocular vision

Abstract: In this paper we present an online approach to segmenting roads on large scale trajectories using only a monocular camera mounted on a car. We differ from popular 2D segmentation solutions which use single colour images and machine learning algorithms that require supervised training on huge image databases. Instead, we propose a novel approach that fuses 3D geometric data with appearance-based segmentation of 2D information in an automatic system. Our contribution is twofold: first, we propagate labels from frame to frame using depth priors of the segmented road avoiding user interaction most of the time; second, we transfer the segmented road labels to 3D laser point clouds. This reduces the complexity of state-of-the-art segmentation algorithms running on 3D Lidar data. Segmentation fails is in only 3% of the cases over a sequence of 13,600 monocular images spanning an urban trajectory of more than 10km.

Vehicle detection and tracking method based on monocular vision

Abstract: The invention requests to protect a vehicle detection and tracking method based on monocular vision, and belongs to the field of machine vision. The method comprises the steps that in a vehicle detection stage, a rod driving region is determined by combining interest region extracting, self-adaptive Canny edge detection and lane line detection, then a vehicle bottom shadow is obtained by means of local tonal value statistics and a thresholding method in which a maximum between-class variance method is used twice in an integrated manner, furthermore, a supposed vehicle is proposed, and then the supposed vehicle is verified through a texture description co-occurrence matrix method; and in a vehicle tracking stage, an improved algorithm combining Kalman filtering with Cam shift is adopted to carry out multi-target tracking, then new target judging, whether searching is successful, and whether a vehicle is out of an edge are used as three standards, and if a new target is detected twice, the new target is processed as a new tracking target, and the tracking target is updated continuously. By adopting the method, vehicle detection and multi-target tracking under a dynamic background are realized, and the instantaneity, the accuracy and the reliability are relatively high.

Autonomous landing of small unmanned aerial rotorcraft based on monocular vision in GPS-denied area

Abstract: Focusing on the low-precision attitude of a current small unmanned aerial rotorcraft at the landing stage, the present paper proposes a new attitude control method for the GPS-denied scenario based on the monocular vision. Primarily, a robust landmark detection technique is developed which leverages the well-documented merits of supporting vector machines (SVMs) to enable landmark detection. Then an algorithm of nonlinear optimization based on Newton iteration method for the attitude and position of camera is put forward to reduce the projection error and get an optimized solution. By introducing the wavelet analysis into the adaptive Kalman filter, the high frequency noise of vision is filtered out successfully. At last, automatic landing tests are performed to verify the method's feasibility and effectiveness.

Semi-Dense Visual Odometry for Monocular Navigation in Clutt ered Environment

Abstract: Recently, there have been numerous advances in the development of biologically inspired lightweight Micro Aerial Vehicles (MAVs). Due to payload and power constraints it is necessary for such systems to have autonomous navigation and flight capabilites in highly dense and cluttered environments using only passive sensors such as cameras. This is a challenging problem, given they have to operate in highly variable illumination conditions and be responsive to large environmental variations. In this paper we describe a scale-aware monocular vision based semi-dense direct depth perception system that enables robust autonomous navigation of small agile MAVs at low altitude through natural forest environments. We also show qualitative results in an outdoor dense forest area.

Influences of monocular and binocular vision on postural stability.

Abstract: To compare the influences of monocular vision versus binocular vision on postural control, twenty-seven otherwise healthy adults, aged from 19 to 38 years, with corrected visual acuity of better than or equal to 20/20, were recruited Body sway for standing 30 seconds on a force platform in 3 conditions was recorded for each participant: one with both eyes open (BEO), one with left eye open (LEO) and the other with both eyes closed (BEC) Postural stability was subsequently evaluated by measuring the total track length (TL) and surface area (SA) of center of pressure of body sway The results show that the values of TL and SA of BEC were significantly greater than those of LEO and BEO Moreover, the values of TL and SA of BEO were significantly smaller than those of LEO (p < 005, one-way repeated measures ANOVA and post hoc of Fisher's LSD procedure) The Romberg coefficient of LEO was also significantly greater than that of BEO (p < 005, paired sample t-test) We concluded that optimal binocular vision provides more information for keeping balance than monocular vision according to the results revealed in our study Assessment of visual acuity is recommended before doing a posturographic test in the clinical setting However, the long-term impact of blindness on controlling posture is uncertain and needs further investigation

Vision-based navigation and guidance of a sensorless missile

Abstract: The objective of this paper is to develop a vision-based terminal guidance system for sensorless missiles. Specifically, monocular vision-based relative navigation and robust control methods are developed for a sensorless missile to intercept a ground target maneuvering with unknown time-varying velocity. A mobile wireless sensor and actor network is considered wherein a moving airborne monocular camera (e.g., attached to an aircraft) provides image measurements of the missile (actor) while another moving monocular camera (e.g., attached to a small UAV) tracks a ground target. The challenge is to express the unknown time-varying target position in the time-varying missile frame using image feedback from cameras moving with unknown trajectories. In a novel relative navigation approach, assuming the knowledge of a single geometric length on the missile, the time-varying target position is obtained by fusing the daisy-chained image measurements of the missile and the target into a homography-based Euclidean reconstruction method. The three-dimensional interception problem is posed in pursuit guidance, proportional navigation, and the proposed hybrid guidance framework. Interestingly, it will be shown that by appropriately defining the error system a single control structure can be maintained across all the above guidance methods. The control problem is formulated in terms of target dynamics in a ‘virtual’ camera mounted on the missile, which enables design of an adaptive nonlinear visual servo controller that compensates for the unknown time-varying missile–target relative velocity. Stability and zero-miss distance analysis of the proposed controller is presented, and a high-fidelity numerical simulation verifies the performance of the guidance laws.