Learning and then recognizing a route, whether travelled during the day or at night, in clear or inclement weather, and in summer or winter is a challenging task for state of the art algorithms in computer vision and robotics. In this paper, we present a new approach to visual navigation under changing conditions dubbed SeqSLAM. Instead of calculating the single location most likely given a current image, our approach calculates the best candidate matching location within every local navigation sequence. Localization is then achieved by recognizing coherent sequences of these “local best matches”. This approach removes the need for global matching performance by the vision front-end - instead it must only pick the best match within any short sequence of images. The approach is applicable over environment changes that render traditional feature-based techniques ineffective. Using two car-mounted camera datasets we demonstrate the effectiveness of the algorithm and compare it to one of the most successful feature-based SLAM algorithms, FAB-MAP. The perceptual change in the datasets is extreme; repeated traverses through environments during the day and then in the middle of the night, at times separated by months or years and in opposite seasons, and in clear weather and extremely heavy rain. While the feature-based method fails, the sequence-based algorithm is able to match trajectory segments at 100% precision with recall rates of up to 60%.

SeqSLAM : visual route-based navigation for sunny summer days and stormy winter nights

Object detection is a fundamental but challenging issue in the field of generic image analysis; it plays an important role in a wide range of applications and has been receiving special attention in recent years. Although there are enomerous methods exist, an in-depth review of the literature concerning generic detection remains. This paper provides a comprehensive survey of recent advances in visual object detection with deep learning. Covering about 300 publications that we survey 1) region proposal-based object detection methods such as R-CNN, SPPnet, Fast R-CNN, Faster R-CNN, Mask RCN, RFCN, FPN, 2) classification/regression base object detection methods such as YOLO(v2 to v5), SSD, DSSD, RetinaNet, RefineDet, CornerNet, EfficientDet, M2Det 3) Some latest detectors such as, relation network for object detection, DCN v2, NAS FPN. Moreover, five publicly available benchmark datasets and their standard evaluation metrics are also discussed. We mainly focus on the application of deep learning architectures to five major applications, namely Object Detection in Surveillance, Military, Transportation, Medical, and Daily Life. In the survey, we cover a variety of factors affecting the detection performance in detail, such as i) a wide range of object categories and intra-class variations, ii) limited storage capacity and computational power. Finally, we finish the survey by identifying fifteen current trends and promising direction for future research.

/pdf/exploring-deep-learning-based-architecture-strategies-4m33u38emf.pdf

Exploring Deep Learning-Based Architecture, Strategies, Applications and Current Trends in Generic Object Detection: A Comprehensive Review

One major concern in the development of intelligent vehicles is to improve the driving safety. It is also an essential issue for future autonomous driving and intelligent transportation. In this paper, we present a vision-based system for driving assistance. A front and a rear on-board camera are adopted for visual sensing and environment perception. The purpose is to avoid potential traffic accidents due to forward collision and vehicle overtaking, and assist the drivers or self-driving cars to perform safe lane change operations. The proposed techniques consist of lane change detection, forward collision warning, and overtaking vehicle identification. A new cumulative density function (CDF)-based symmetry verification method is proposed for the detection of front vehicles. The motion cue obtained from optical flow is used for overtaking detection. It is further combined with a convolutional neural network to remove repetitive patterns for more accurate overtaking vehicle identification. Our approach is able to adapt to a variety of highway and urban scenarios under different illumination conditions. The experiments and performance evaluation carried out on real scene images have demonstrated the effectiveness of the proposed techniques.

A Vision-Based Driver Assistance System with Forward Collision and Overtaking Detection

Due to the increasing number of vehicles in circulation in different urban cities, several automatic traffic monitoring systems have been developed. In particular, traffic monitoring systems using roadside cameras are becoming extensively deployed, as they offer imperative technological advantages compared with other traffic monitoring systems. Vehicle detection and traffic congestion classification are two main steps for video‐based traffic congestion detection systems; the associated methods have a deep impact on the performance of the whole system. In this paper, we investigate four selected vehicle detection methods namely Gaussian Mixture Model (GMM), GMM‐Kalman filter, Optical Flow, and ACF object detector in two contexts: urban and highway. Three traffic congestion classification methods are also studied. The comparative study of the different methods allows us to choose the most appropriate ones to be integrated in the framework proposed to solve the traffic issues in the bridge of Bizerte.

Vision‐based vehicle detection for road traffic congestion classification

Real-time, accurate, and robust localisation is critical for autonomous vehicles (AVs) to achieve safe, efficient driving, whilst real-time performance is essential for AVs to achieve their current position in time for decision making. To date, no review paper has quantitatively compared the real-time performance between different localisation techniques based on various hardware platforms and programming languages and analysed the relations among localisation methodologies, real-time performance and accuracy. Therefore, this paper discusses the state-of-the-art localisation techniques and analyses their overall performance in AV application. For further analysis, this paper firstly proposes a localisation algorithm operations capability (LAOC)-based equivalent comparison method to compare the relative computational complexity of different localisation techniques; then, it comprehensively discusses the relations among methodologies, computational complexity, and accuracy. Analysis results show that the computational complexity of localisation approaches differs by a maximum of about 10⁷ times, whilst accuracy varies by about 100 times. Vision- and data fusion-based localisation techniques have about 2-5 times potential for improving accuracy compared with lidar-based localisation. Lidar- and vision-based localisation can reduce computational complexity by improving image registration method efficiency. Data fusion-based localisation can achieve better real-time performance compared with lidar- and vision-based localisation because each standalone sensor does not need to develop a complex algorithm to achieve its best localisation potential. Vehicle-to-everything (V2X) technology can improve positioning robustness. Finally, the potential solutions and future orientations of AVs' localisation based on the quantitative comparison results are discussed.

/pdf/real-time-performance-focused-on-localisation-techniques-for-edh46qwevh.pdf

Real-time performance-focused on localisation techniques for autonomous vehicle: a review

Measuring the distance is one of the main objectives for sensor development. In this article, we present a new sensor system design for depth recovery. It is based on stereopsis with the integrated rotation of an image sensor. Different from conventional stereo vision systems with multiple cameras, the proposed rotational stereo configuration is built upon a single imaging device. By directing its lateral motion with a circular constraint, the visual parallax is generated from multiple viewpoints. The image pairs captured with the integrated sensor rotation are used for the correspondence matching and disparity computation. Moreover, a variation of multiple-baseline stereo with multiple base-angle implementation is proposed. It provides a flexible baseline setting to improve the depth measurement accuracy. The geometric formulation and computational algorithm are derived and developed. In the experiments, a prototype range sensing device is constructed. The results obtained from the real scene environment have demonstrated the feasibility of the proposed depth measurement approach.

Depth Measurement Based on Stereo Vision With Integrated Camera Rotation

The ability to reliably measure the depth of the object surface is very important in a range of high-value industries. With the development of 3D vision techniques, RGB-D cameras have been widely used to perform the 6D pose estimation of target objects for a robotic manipulator. Many applications require accurate shape measurements of the objects for 3D template matching. In this work, we develop an RGB-D camera based on the structured light technique with gray-code coding. The intrinsic and extrinsic parameters of the camera system are determined by a calibration process. 3D reconstruction of the object surface is based on the ray triangulation principle. We construct an RGB-D sensing system with an industrial camera and a digital light projector. In the experiments, real-world objects are used to test the feasibility of the proposed technique. The evaluation carried out using planar objects has demonstrated the accuracy of our RGB-D depth measurement system.

/pdf/a-structured-light-rgb-d-camera-system-for-accurate-depth-n4klnjer12.pdf

A Structured Light RGB-D Camera System for Accurate Depth Measurement

This paper presents a vehicle localization method to assist vehicle navigation based on topological map construction and scene recognition. A topological map is constructed using omni-directional image sequences, and the node information of the topological map is used for place recognition and derivation of vehicle location. In topological map construction and scene change detection, we utilize the Extended-HCT method for semantic description and feature extraction. Content-based and feature-based image retrieval approaches are adopted for place recognition and vehicle localization on the real scene image dataset. The proposed technique is able to construct a real-time image retrieval system for navigation assistance and validate the correctness of the route. Experiments are carried out in both the indoor and outdoor environments using real world images.

Topological map construction and scene recognition for vehicle localization

We propose an enhanced approach for recognizing traffic signs; images are captured using a video camera installed in a car. Support vector machines detect the edges and gradients of each captured i...

Improved traffic sign recognition for in-car cameras

Deep learning methods, especially multi-task learning with CNNs, have achieved good results in many fields of computer vision. Semantic segmentation and shape detection of lumbar vertebrae, sacrum, and femoral heads from clinical X-ray images are important and challenging tasks. In this paper, we propose a multi-task deep neural network, MBNet. It is developed based on our new multi-path convolutional neural network, BiLuNet, for semantic segmentation on X-ray images. Our MBNet has two branches, one is for semantic segmentation of lumbar vertebrae, sacrum, and femoral heads. It shares the main features with the second branch to learn and classify by supervised learning. The output of the second branch is to predict the inspected values for lumbar vertebra inspection. These networks are capable of performing the two tasks with very limited training data. We collected our dataset and annotated it by doctors for model training and performance evaluation. Compared to the state-of-the-art methods, our BiLuNet model provides better mIoUs with the same training data. The experimental results have demonstrated the feasibility of our MBNet for semantic segmentation of lumbar vertebrae, as well as the parameter prediction for the doctors to perform clinical diagnosis of low back pains. Code is available at: https://github.com/LuanTran07/BiLUnet-Lumbar-Spine.

/pdf/mbnet-a-multi-task-deep-neural-network-for-semantic-4q2yz13odk.pdf

Van Luan Tran

Papers

Depth Measurement Based on Stereo Vision With Integrated Camera Rotation

A Structured Light RGB-D Camera System for Accurate Depth Measurement

Topological map construction and scene recognition for vehicle localization

Improved traffic sign recognition for in-car cameras

MBNet: A Multi-task Deep Neural Network for Semantic Segmentation and Lumbar Vertebra Inspection on X-Ray Images