The safe and efficient autonomous navigation of mobile manipulators is still challenging in harsh manufacturing environments with dynamic obstacles and narrow spaces. This paper addresses that challenge by proposing an industrial-grade LIDAR-based navigation system for four-wheel-steering and four-wheel-driving mobile manipulators and focuses on enhancing the system’s safety and efficiency in manufacturing plants. On one hand, using an efficient soft-evidence unscented distance filter, a robust localization method is presented to eliminate dynamic interference. The variation results of scan matching are ingeniously applied to formulate a practical localization-failure-detection strategy, therefore improving localization safety under confusing working conditions. On the other hand, an efficient path-graph-based trajectory-planning method is proposed, which can instantly generate a trajectory in a safe and effective search space. Considering the regional connectivity and movement characteristics of the 4WS4WD mobile manipulator, a novel path graph is designed to construct the search space. Even in a large-scale scenario, a robot can obtain the desired trajectory within 5 ms. Ultimately, some core modules, such as a safety module and a fault-tolerant control module are integrated to form a comprehensive navigation system. The simulated and real experimental results prove the excellent performance of our navigation system, which has been successfully applied in many real-world scenarios.

https://iopscience.iop.org/article/10.1088/1361-6501/abce71/pdf

A safe and efficient LIDAR-based navigation system for 4WS4WD mobile manipulators in manufacturing plants

Trajectory data are often used as important auxiliary information in preprocessing and extracting the target from mobile laser scanning data. However, the trajectory data stored independently may be lost and destroyed for various reasons, making the data unavailable for the relevant models. This study proposes recovering the trajectory of the scanner from point cloud data following the scanning principles of a rotating mirror. Two approaches are proposed from different input conditions: Ordered three-dimensional coordinates of point cloud data, with and without acquisition time. We recovered the scanner’s ground track through road point density analysis and restored the position of the center of emission of the laser based on plane reconstruction on a single scanning line. The validity and reliability of the proposed approaches were verified in the four typical urban, rural, winding, and viaduct road environments using two systems from different manufacturers. The result deviations of the ground track and scanner trajectory from their actual position were a few centimeters and less than 1 decimeter, respectively. Such an error is sufficiently small for the trajectory data to be used in the relevant algorithms.

https://www.mdpi.com/2072-4292/12/6/899/pdf

Recovering Missing Trajectory Data for Mobile Laser Scanning Systems

Binary Neural Networks (BNNs) are showing tremen-dous success on realistic image classification tasks. Notably, their accuracy is similar to the state-of-the-art accuracy obtained by full-precision models tailored to edge devices. In this regard, BNNs are very amenable to edge devices since they employ 1-bit to store the inputs and weights, and thus, their storage requirements are low. Moreover, BNNs computations are mainly done using xnor and pop-counts operations which are implemented very efficiently using simple hardware structures. Nonetheless, supporting BNNs efficiently on mobile CPUs is far from trivial since their benefits are hindered by frequent memory accesses to load weights and inputs. In BNNs, a weight or an input is stored using one bit, and aiming to increase storage and computation efficiency, several of them are packed together as a sequence of bits. In this work, we observe that the number of unique sequences representing a set of weights or inputs is typically low (i.e., 512). Also, we have seen that during the evaluation of a BNN layer, a small group of unique sequences is employed more frequently than others. Accordingly, we propose exploiting this observation by using Huffman Encoding to encode the bit sequences and then using an indirection table to decode them during the BNN evaluation. Also, we propose a clustering-based scheme to identify the most common sequences of bits and replace the less common ones with some similar common sequences. As a result, we decrease the storage requirements and memory accesses since the most common sequences are encoded with fewer bits. In this work, we extend a mobile CPU by adding a small hardware structure that can efficiently cache and decode the compressed sequence of bits. We evaluate our scheme using the ReAacNet model with the Imagenet dataset on an ARM CPU. Our experimental results show that our technique can reduce memory requirement by 1.32x and improve performance by 1.35x. 

/pdf/exploiting-kernel-compression-on-bnns-24ijy96d.pdf

Exploiting Kernel Compression on BNNs

Binary Neural Networks (BNNs) are showing tremen-dous success on realistic image classification tasks. Notably, their accuracy is similar to the state-of-the-art accuracy obtained by full-precision models tailored to edge devices. In this regard, BNNs are very amenable to edge devices since they employ 1-bit to store the inputs and weights, and thus, their storage requirements are low. Moreover, BNNs computations are mainly done using xnor and pop-counts operations which are implemented very efficiently using simple hardware structures. Nonetheless, supporting BNNs efficiently on mobile CPUs is far from trivial since their benefits are hindered by frequent memory accesses to load weights and inputs. In BNNs, a weight or an input is stored using one bit, and aiming to increase storage and computation efficiency, several of them are packed together as a sequence of bits. In this work, we observe that the number of unique sequences representing a set of weights or inputs is typically low (i.e., 512). Also, we have seen that during the evaluation of a BNN layer, a small group of unique sequences is employed more frequently than others. Accordingly, we propose exploiting this observation by using Huffman Encoding to encode the bit sequences and then using an indirection table to decode them during the BNN evaluation. Also, we propose a clustering-based scheme to identify the most common sequences of bits and replace the less common ones with some similar common sequences. As a result, we decrease the storage requirements and memory accesses since the most common sequences are encoded with fewer bits. In this work, we extend a mobile CPU by adding a small hardware structure that can efficiently cache and decode the compressed sequence of bits. We evaluate our scheme using the ReAacNet model with the Imagenet dataset on an ARM CPU. Our experimental results show that our technique can reduce memory requirement by 1.32x and improve performance by 1.35x.

As systems that utilize computer vision move into the public domain, methods of calibration need to become easier to use. Though multi-plane LiDAR systems have proven to be useful for vehicles and large robotic platforms, many smaller platforms and low-cost solutions still require 2D LiDAR combined with RGB cameras. Current methods of calibrating these sensors make assumptions about camera and laser placement and/or require complex calibration routines. In this paper we propose a new method of feature correspondence in the two sensors and an optimization method capable of using a calibration target with unknown lengths in its geometry. Our system is designed with an inexperienced layperson as the intended user, which has led us to remove as many assumptions about both the target and laser as possible. We show that our system is capable of calibrating the 2-sensor system from a single sample in configurations other methods are unable to handle.

Simple Camera-to-2D-LiDAR Calibration Method for General Use

Two dimensional (2D) laser rangefinders (LRF) are widely utilized in various mobilized systems, such as robot, intelligent vehicles (IVs), and mobile mapping. Pose estimation for a 2D LRF has important applications for exterior calibration of multiple sensors, registering light detection and ranging (LIDAR) data from multiple LRFs or from a spinning/nodding LRF, etc. This paper shows that the pose of a 2D LRF can be uniquely determined from the single-shot of a trirectangular trihedron. The pose is estimated by solving a simplified Perspective-Three-Point (P3P) problem and a Three-Point-Registration problem. Compared to the raw noisy LIDAR data, the three control points in P3P problem are estimated from line fitting of the sensed LIDAR data to guarantee good and reliable estimation results. The proposed method has been validated with both simulation and real data. The results show that the method is accurate and practical. To the best of our knowledge, this paper solves the pose estimation of a 2D LRF by using LIDAR data of one scan for the first time.

A single-shot pose estimation approach for a 2D laser rangefinder

Binarized neural networks (BNNs) and batch normalization (BN) have already become typical techniques in artificial intelligence today. Unfortunately, the massive accumulation and multiplication in BNN models bring challenges to field-programmable gate array (FPGA) implementations, because complex arithmetics in BN consume too much computing resources. To relax FPGA resource limitations and speed up the computing process, we propose a BNN accelerator architecture based on consolidation compressed tree scheme by combining both XNOR and accumulation operation of the low bit into a systematic one. During the compression process, we adopt 0-padding (not ±1) to achieve no-accuracy-loss from software modeling to hardware implementation. Moreover, we introduce shift-addition-BN free binarization technique to shorten the delay path and optimize on-chip storage. To sum up, we drastically cut down the hardware consumption while maintaining great speed performance with the same model complexity as the previous design. We evaluate our accelerator on MNIST and CIFAR-10 dataset and implement the whole system on the ARTIX-7 100T FPGA with speed performance of 2052.65 GOP/s and area efficiency of 70.15 GOPS/KLUT.

Junjie Zhou

Papers

A single-shot pose estimation approach for a 2D laser rangefinder

A BNN Accelerator Based on Edge-skip-calculation Strategy and Consolidation Compressed Tree

Infrared blind spectral deconvolution with low-rank sparse regularization for small object tracking