Showing papers by "Ivan Petrović published in 2021"

Spatiotemporal Multisensor Calibration via Gaussian Processes Moving Target Tracking

Abstract: Robust and reliable perception of autonomous systems often relies on fusion of heterogeneous sensors, which poses great challenges for multisensor calibration. In this article, we propose a method for multisensor calibration based on Gaussian processes (GPs) estimated moving target trajectories, resulting with spatiotemporal calibration. Unlike competing approaches, the proposed method is characterized by the following: first, joint multisensor on-manifold spatiotemporal optimization framework, second, batch state estimation and interpolation using GPs, and, third, computational efficiency with O(n) complexity. It only requires that all sensors can track the same target. The method is validated in simulation and real-world experiments on the following five different multisensor setups: first, hardware triggered stereo camera, second, camera and motion capture system, third, camera and automotive radar, fourth, camera and rotating 3-D lidar, and, fifth, camera, 3-D lidar, and the motion capture system. The method estimates time delays with the accuracy up to a fraction of the fastest sensor sampling time, outperforming a state-of-the-art ego-motion method. Furthermore, this article is complemented by an open-source toolbox implementing the calibration method available at bitbucket.org/unizg-fer-lamor/calirad.

A Continuous-Time Approach for 3D Radar-to-Camera Extrinsic Calibration

Abstract: Reliable operation in inclement weather is essential to the deployment of safe autonomous vehicles (AVs). Robustness and reliability can be achieved by fusing data from the standard AV sensor suite (i.e., lidars, cameras) with weather robust sensors, such as millimetre-wavelength radar. Critically, accurate sensor data fusion requires knowledge of the rigidbody transform between sensor pairs, which can be determined through the process of extrinsic calibration. A number of extrinsic calibration algorithms have been designed for 2D (planar) radar sensors—however, recently-developed, low-cost 3D millimetre-wavelength radars are set to displace their 2D counterparts in many applications. In this paper, we present a continuous-time 3D radar-to-camera extrinsic calibration algorithm that utilizes radar velocity measurements and, unlike the majority of existing techniques, does not require specialized radar retroreflectors to be present in the environment. We derive the observability properties of our formulation and demonstrate the efficacy of our algorithm through synthetic and real-world experiments.

Online Multi-Sensor Calibration Based on Moving Object Tracking

Abstract: Modern autonomous systems often fuse information from many different sensors to enhance their perception capabilities. For successful fusion, sensor calibration is necessary, while performing it on...

Drivable Path Planning Using Hybrid Search Algorithm Based on E* and Bernstein–Bézier Motion Primitives

Abstract: This article proposes a hybrid path-planning algorithm, the HE* algorithm, which combines the discrete grid-based E* search and continuous Bernstein–Bezier (BB) motion primitives. Several researchers have addressed the smooth path planning problem and the sample-based integrated path planning techniques. We believe that the main benefits of the proposed approach are: directly drivable path, no additional post-optimization tasks, reduced search branching, low computational complexity, and completeness guarantee. Several examples and comparisons with the state-of-the-art planners are provided to illustrate and evaluate the main advantages of the HE* algorithm. HE* yields a collision-safe and smooth path that is close to spatially optimal (the Euclidean shortest path) with a guaranteed continuity of curvature. Therefore, the path is easily drivable for a wheeled robot without any additional post-optimization and smoothing required. HE* is a two-stage algorithm which uses a direction-guiding heuristics computed by the E* search in the first stage, which improves the quality and reduces the complexity of the hybrid search in the second stage. In each iteration, the search is expanded by a set of BBs, the parameters of which adapt continuously according to the guiding heuristics. Completeness is guaranteed by relying on a complete node mechanism, which also provides an upper bound for the calculated path cost. A remarkable feature of HE* is that it produces good results even at coarse resolutions.

Recalibrating the KITTI Dataset Camera Setup for Improved Odometry Accuracy

Abstract: Over the last decade, one of the most relevant public datasets for evaluating odometry accuracy is the KITTI dataset. Beside the quality and rich sensor setup, its success is also due to the online evaluation tool, which enables researchers to bench-mark and compare algorithms. The results are evaluated on the test subset solely, without any knowledge about the ground truth, yielding unbiased, overfit free and therefore relevant validation for robot localization based on cameras, 3D laser or combination of both. However, as any sensor setup, it requires prior calibration and rectified stereo images are provided, introducing dependence on the default calibration parameters. Given that, a natural question arises if a better set of calibration parameters can be found that would yield higher odometry accuracy. In this paper, we propose a new approach for one shot calibration of the KITTI dataset multiple camera setup. The approach yields better calibration parameters, both in the sense of lower calibration reprojection errors and lower visual odometry error. We conducted experiments where we show for three different odometry algorithms, namely SOFT2, ORB-SLAM2 and VISO2, that odometry accuracy is significantly improved with the pro-posed calibration parameters. Moreover, our odometry, SOFT2, in conjunction with the proposed calibration method achieved the highest accuracy on the official KITTI scoreboard with 0.53% translational and 0.0009 deg/m rotational error, outperforming even 3D laser-based methods.

Stereo dense depth tracking based on optical flow using frames and events

Abstract: Event cameras are biologically inspired sensors that asynchronously detect brightness changes in the scene independently for each pixel. Their output is a stream of events which is reported with a ...

A Comparison of Graph Optimization Approaches for Pose Estimation in SLAM

Abstract: Simultaneous localization and mapping (SLAM) is an important tool that enables autonomous navigation of mobile robots through unknown environments. As the name SLAM suggests, it is important to obtain a correct representation of the environment and estimate a correct trajectory of the robot poses in the map. Dominant state-of-the-art approaches solve the pose estimation problem using graph optimization techniques based on the least squares minimization method. Among the most popular approaches are libraries such as g2o, Ceres, GTSAM and SE-Sync. The aim of this paper is to describe these approaches in a unified manner and to evaluate them on an array of publicly available synthetic and real-world pose graph datasets. In the evaluation experiments, the computation time and the value of the objective function of the four optimization libraries are analyzed.

Active vision for 3D indoor scene reconstruction using a 3D camera on a pan-tilt mechanism

Abstract: We present a novel approach to automatic indoor scene reconstruction from RGB-D images acquired from a single viewpoint using active vision. The proposed method is designed to select the next view ...

Extended Kalman filter for payload state estimation utilizing aircraft inertial sensing

Abstract: In this paper we consider an aerial vehicle transporting a suspended payload and propose an Extended Kalman filter for payload state estimation. The filter is based on derived system dynamics and relies solely on onboard IMU measurements. Effectiveness of the method is verified in numerical simulations and experimentally.

Mobile Robot Teleoperation via Android Mobile Device with UDP Communication

Abstract: Teleoperation is an essential component of a robotic system utilized for execution of various remote tasks, e.g. telepresence, reconnaissance or search and rescue. In this paper we investigate the usage of User Datagram Protocol (UDP) for two-way communication between an Android mobile device and Robot Operating System (ROS). UDP offers simplified and connectionless communication, which allows high transmission rates and enables real-time applications. We develop an Android application which turns omnipresent Android devices into cheap and intuitive robot teleoperation interfaces. It enables interpreting user's desired movement choices into corresponding datagrams, which are then sent to the robot via wireless network. As a case study, we implement a solution for controlling the Turtlebot3 differential drive mobile robot within ROS. We examine the utility of UDP-based teleoperation in both simulation and real-life environment. We complement the paper with modular open-source implementations of our Android application and a ROS package featuring the Turtlebot3 controller.

Multi-Target Tracking on Riemannian Manifolds via Probabilistic Data Association

Abstract: Riemannian manifolds are attracting much interest in various technical disciplines, since generated data can often be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of such manifolds, usual Euclidean methods yield inferior results, thus motivating development of tools adapted or specially tailored to the true underlying geometry. In this letter we propose a method for tracking multiple targets residing on smooth manifolds via probabilistic data association. By using tools of differential geometry, such as exponential and logarithmic mapping along with the parallel transport, we extend the Euclidean multi-target tracking techniques based on probabilistic data association to systems constrained to a Riemannian manifold. The performance of the proposed method was extensively tested in experiments simulating multi-target tracking on unit hyperspheres, where we compared our approach to the von Mises-Fisher and the Kalman filters in the embedding space that projects the estimated state back to the manifold. Obtained results show that the proposed method outperforms the competitive trackers in the optimal sub-pattern assignment metric for all the tested hypersphere dimensions. Although our use case geometry is that of a unit hypersphere, our approach is by no means limited to it and can be applied to any Riemannian manifold with closed-form expressions for exponential/logarithmic maps and parallel transport along the geodesic curve. The paper code is publicly available 1 .

A Continuous-Time Approach for 3D Radar-to-Camera Extrinsic Calibration

Abstract: Reliable operation in inclement weather is essential to the deployment of safe autonomous vehicles (AVs). Robustness and reliability can be achieved by fusing data from the standard AV sensor suite (i.e., lidars, cameras) with weather robust sensors, such as millimetre-wavelength radar. Critically, accurate sensor data fusion requires knowledge of the rigid-body transform between sensor pairs, which can be determined through the process of extrinsic calibration. A number of extrinsic calibration algorithms have been designed for 2D (planar) radar sensors - however, recently-developed, low-cost 3D millimetre-wavelength radars are set to displace their 2D counterparts in many applications. In this paper, we present a continuous-time 3D radar-to-camera extrinsic calibration algorithm that utilizes radar velocity measurements and, unlike the majority of existing techniques, does not require specialized radar retroreflectors to be present in the environment. We derive the observability properties of our formulation and demonstrate the efficacy of our algorithm through synthetic and real-world experiments.

Feature-based Event Stereo Visual Odometry

Abstract: Event-based cameras are biologically inspired sensors that output events, i.e., asynchronous pixel-wise brightness changes in the scene. Their high dynamic range and temporal resolution of a microsecond makes them more reliable than standard cameras in environments of challenging illumination and in high-speed scenarios, thus developing odometry algorithms based solely on event cameras offers exciting new possibilities for autonomous systems and robots. In this paper, we propose a novel stereo visual odometry method for event cameras based on feature detection and matching with careful feature management, while pose estimation is done by feature reprojection error minimization. We evaluate the performance of the proposed method on two publicly available datasets: MVSEC sequences captured by an indoor flying drone and DSEC outdoor driving sequences. MVSEC offers accurate ground truth from motion capture, while for DSEC, which does not offer ground truth, in order to obtain a reference trajectory on the standard camera frames we used our SOFT visual odometry, one of the highest ranking algorithms on the KITTI scoreboards. We compared our method to the ESVO method, which is the first and still the only stereo event odometry method, showing on par performance on both MVSEC and DSEC sequences. Furthermore, two important advantages of our method over ESVO are that it adapts tracking frequency to the asynchronous event rate and does not require initialization.

Ensemble of LSTMs and feature selection for human action prediction

Abstract: As robots are becoming more and more ubiquitous in human environments, it will be necessary for robotic systems to better understand and predict human actions. However, this is not an easy task, at times not even for us humans, but based on a relatively structured set of possible actions, appropriate cues, and the right model, this problem can be computationally tackled. In this paper, we propose to use an ensemble of long-short term memory (LSTM) networks for human action prediction. To train and evaluate models, we used the MoGaze dataset - currently the most comprehensive dataset capturing poses of human joints and the human gaze. We have thoroughly analyzed the MoGaze dataset and selected a reduced set of cues for this task. Our model can predict (i) which of the labeled objects the human is going to grasp, and (ii) which of the macro locations the human is going to visit (such as table or shelf). We have exhaustively evaluated the proposed method and compared it to individual cue baselines. The results suggest that our LSTM model slightly outperforms the gaze baseline in single object picking accuracy, but achieves better accuracy in macro object prediction. Furthermore, we have also analyzed the prediction accuracy when the gaze is not used, and in this case, the LSTM model considerably outperformed the best single cue baseline

Depth from Mono Accuracy Analysis by Changing Camera Parameters in the CARLA Simulator

Abstract: Depth estimation is an important task in robotics and autonomous driving. By estimating depth and relying only on a single camera, it is no longer necessary to add and calibrate additional sensors - usually a second camera. However, such an approach requires training on extensive datasets and obtaining real-world datasets is time consuming and costly. Given that, using photorealistic simulators can be beneficial, since a multitude of varoius scenes can be created. In this paper we present an approach to training a deep neural network based on the ResNet architecture for estimating depth from a single camera. We target road vehicle scenes and use the CARLA simulator. We evaluate the trained network on the real-world KITTI dataset images and in the CARLA simulator. In the simulated experiments, we compare the performance with respect to the changes in camera intrinsic and extrinsic calibration parameters with respect to the ego vehicle frame.

Recalibrating the KITTI Dataset Camera Setup for Improved Odometry Accuracy

Abstract: Over the last decade, one of the most relevant public datasets for evaluating odometry accuracy is the KITTI dataset. Beside the quality and rich sensor setup, its success is also due to the online evaluation tool, which enables researchers to benchmark and compare algorithms. The results are evaluated on the test subset solely, without any knowledge about the ground truth, yielding unbiased, overfit free and therefore relevant validation for robot localization based on cameras, 3D laser or combination of both. However, as any sensor setup, it requires prior calibration and rectified stereo images are provided, introducing dependence on the default calibration parameters. Given that, a natural question arises if a better set of calibration parameters can be found that would yield higher odometry accuracy. In this paper, we propose a new approach for one shot calibration of the KITTI dataset multiple camera setup. The approach yields better calibration parameters, both in the sense of lower calibration reprojection errors and lower visual odometry error. We conducted experiments where we show for three different odometry algorithms, namely SOFT2, ORB-SLAM2 and VISO2, that odometry accuracy is significantly improved with the proposed calibration parameters. Moreover, our odometry, SOFT2, in conjunction with the proposed calibration method achieved the highest accuracy on the official KITTI scoreboard with 0.53% translational and 0.0009 deg/m rotational error, outperforming even 3D laser-based methods.

Riemannian Optimization for Distance Geometric Inverse Kinematics

Abstract: Solving the inverse kinematics problem is a fundamental challenge in motion planning, control, and calibration for articulated robots. Kinematic models for these robots are typically parameterized by joint angles, generating a complicated mapping between a robot's configuration and end-effector pose. Alternatively, the kinematic model and task constraints can be represented using invariant distances between points attached to the robot. In this paper, we formalize the equivalence of distance-based inverse kinematics and the distance geometry problem for a large class of articulated robots and task constraints. Unlike previous approaches, we use the connection between distance geometry and low-rank matrix completion to find inverse kinematics solutions by completing a partial Euclidean distance matrix through local optimization. Furthermore, we parameterize the space of Euclidean distance matrices with the Riemannian manifold of fixed-rank Gram matrices, allowing us to leverage a variety of mature Riemannian optimization methods. Finally, we show that bound smoothing can be used to generate informed initializations without significant computational overhead, improving convergence. We demonstrate that our novel inverse kinematics solver achieves higher success rates than traditional techniques, and significantly outperforms them on problems that involve many workspace constraints.

Simulation of an Electric Arc in Metal Welding

Abstract: This paper describes the proces of electric arc welding in the zone of protective gas, with an emphasis on transmitting metal dropelts via a short ciruit.Parameters of the welding proces and of an electric arc are described, and a mathemacal model for the MIG system is sugessted. Model simulation is completed used Matlab/Simulink package. The proces is simulated using real parameters in order for the output values of the process to corespond to real values obtained by the means of mesasurement.