Showing papers on "Particle filter published in 2020"
TL;DR: A detailed model for the IPGS is presented with the consideration of the power-to-gas devices and gas storages, and the gas storage life reliability model is considered to characterize the charging and discharging performance.
Abstract: The reliability evaluation of integrated power-gas systems (IPGS) becomes critical due to the high dependency of the two energy systems. Once a contingent incident happens in one system, the other system will accordingly be affected. In this paper, a detailed model for the IPGS is presented with the consideration of the power-to-gas devices and gas storages. Furthermore, a sequential Monte Carlo (SMC) simulation is utilized to evaluate the reliability of the IPGS. In particular, the gas storage life reliability model is considered to characterize the charging and discharging performance. Moreover, an optimal load shedding model is used to coordinate the load shedding of the IPGS. What's more, new reliability indices are given to display the reliability of the IPGS. Finally, the proposed model is tested on an integrated IEEE 24-bus power system and 20-node gas system and an integrated IEEE RTS 96 power system and 40-node gas system. The results show the effectiveness of the proposed model.
119 citations
TL;DR: Backward smoothing square root cubature Kalman filter (BS-SRCKF) is proposed to improve accuracy and convergence speed of SOC estimation and improved cuckoo search (ICS) algorithm is embedded in the standard particle filter (PF) to improve its performance.
95 citations
TL;DR: A new particle swarm optimization particle filter algorithm (NPSO-PF) is proposed, which is called particle cluster optimization particles filter algorithm with mutation operator, and it is proposed that the particle filter can be tuned according to the number of particles in a swarm.
Abstract: In this paper, a new particle swarm optimization particle filter (NPSO-PF) algorithm is proposed, which is called particle cluster optimization particle filter algorithm with mutation operator, and...
95 citations
TL;DR: A modified Kalman Filter (KF) for localization based on UKF and PF Localization algorithms that can be applied to any type of localization approach, especially in the case of robot localization.
Abstract: Localization plays an important role in the field of Wireless Sensor Networks (WSNs) and robotics. Currently, localization is a very vibrant scientific research field with many potential applications. Localization offers a variety of services for the customers, for example, in the field of WSN, its importance is unlimited, in the field of logistics, robotics, and IT services. Particularly localization is coupled with the case of human-machine interaction, autonomous systems, and the applications of augmented reality. Also, the collaboration of WSNs and distributed robotics has led to the creation of Mobile Sensor Networks (MSNs). Nowadays there has been an increasing interest in the creation of MSNs and they are the preferred aspect of WSNs in which mobility plays an important role while an application is going to execute. To overcome the issues regarding localization, the authors developed a framework of three algorithms named Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF) and Particle Filter (PF) Localization algorithms. In our previous study, the authors only focused on EKF-based localization. In this paper, the authors present a modified Kalman Filter (KF) for localization based on UKF and PF Localization. In the paper, all these algorithms are compared in very detail and evaluated based on their performance. The proposed localization algorithms can be applied to any type of localization approach, especially in the case of robot localization. Despite the harsh physical environment and several issues during localization, the result shows an outstanding localization performance within a limited time. The robustness of the proposed algorithms is verified through numerical simulations. The simulation results show that proposed localization algorithms can be used for various purposes such as target tracking, robot localization, and can improve the performance of localization.
86 citations
03 Apr 2020
TL;DR: Particle Filter Recurrent Neural Networks (PF-RNNs) as mentioned in this paper is a new RNN family that explicitly models uncertainty in its internal structure, where a particle filter is used to update the latent state distribution according to the Bayes rule.
Abstract: Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and multi-modal real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.
63 citations
TL;DR: In this article, a mobile application is developed along with three Bayesian filtering techniques to improve the BLE beacon proximity estimation accuracy, including a Kalman filter, a particle filter, and a nonparametric information (NI) filter.
Abstract: The interconnectedness of all things is continuously expanding which has allowed every individual to increase their level of interaction with their surroundings. Internet of Things (IoT) devices are used in a plethora of context-aware application, such as proximity-based services (PBSs), and location-based services (LBSs). For these systems to perform, it is essential to have reliable hardware and predict a user’s position in the area with high accuracy in order to differentiate between individuals in a small area. A variety of wireless solutions that utilize received signal strength indicators (RSSIs) have been proposed to provide PBS and LBS for indoor environments, though each solution presents its own drawbacks. In this article, Bluetooth low energy (BLE) beacons are examined in terms of their accuracy in proximity estimation. Specifically, a mobile application is developed along with three Bayesian filtering techniques to improve the BLE beacon proximity estimation accuracy. This includes a Kalman filter, a particle filter, and a nonparametric information (NI) filter. Since the RSSI is heavily influenced by the environment, experiments were conducted to examine the performance of beacons from three popular vendors in two different environments. The error is compared in terms of mean absolute error (MAE) and root mean squared error (RMSE). According to the experimental results, Bayesian filters can improve proximity estimation accuracy up to 30% in comparison with traditional filtering, when the beacon and the receiver are within 3 m.
56 citations
TL;DR: In this paper, an unbiased estimator of smoothing is proposed for state-space models with noisy measurements related to the process, and the estimator is shown to be unbiased.
Abstract: In state–space models, smoothing refers to the task of estimating a latent stochastic process given noisy measurements related to the process. We propose an unbiased estimator of smoothing expectat...
54 citations
TL;DR: This study aims to introduce the ideas of marking (marking the target by user (observer) in the first frame of a video sequence) and decreasing image size and the performance of the offered RPFGA method to tackle the occlusion problem is enhanced by the marking idea.
Abstract: The particle filter (PF) is an influential instrument for visual tracking; it relies on the Monte Carlo Chain Framework and Bayesian probability that is of tremendous importance for smart monitoring systems. The current study introduces a particle filter based upon genetic resampling. In the suggested method called Reduced Particle Filter based upon Genetic Algorithm (RPFGA), particles with the highest weights are chosen and go through evolution using a GA in the resampling phase of PF algorithm. Moreover, this study aims to introduce the ideas of marking (marking the target by user (observer) in the first frame of a video sequence) and decreasing image size. Applying both ideas leads to reduced number of particles, the processing time of each frame, and the total tracking time. Additionally, the performance of the offered RPFGA method to tackle the occlusion problem is enhanced by the marking idea. According to the results obtained in challenges, such as Occlusions (OCC), deformation (DEF), low resolution (LR), scale variations(SV), Fast Motions (FM), In-Plane Rotation (IPR), Out-Of-Plane Rotation (OPR), Motion Blur (MB), Illumination Variation (IV) and color similarity between the target and the background, and regarding precision and tracking time, the recommended hybrid approach only with a few particles overtakes the generic particle filter, Particle Swarm Optimization particle filter (PSO-PF) and the particle filter based upon improved cuckoo search (ICS-PF). The suggested method can be applied for real time video objects tracking.
49 citations
TL;DR: An event-based distributed filtering scheme is designed, which is an energy-efficient way to transmit data between sensors and estimators and an optimal control problem is formed to control the position of sensors so that the target tracking process can be achieved quickly.
Abstract: In this paper, the distributed remote state estimation problem for conditional dynamic linear systems in mobile sensor networks with an event-triggered mechanism is investigated. The distributed mixture Kalman filtering method is proposed to track the state of the maneuvering target, which uses particle filtering to estimate the nonlinear variables and apply Kalman filtering to estimate the linear variables. An event-based distributed filtering scheme is designed, which is an energy-efficient way to transmit data between sensors and estimators. In addition, by using the mutual information theory, an optimal control problem is formed to control the position of sensors so that the target tracking process can be achieved quickly. Finally, a simulation example about the maneuvering target tracking is provided to corroborate the effectiveness of the filtering method and the control performance for sensors.
46 citations
TL;DR: A constrained mixture sequential Monte Carlo (CMSMC) method in which a mixture representation is incorporated in the estimated posterior distribution to maintain multi-modality to overcome challenges for multi-target tracking.
Abstract: Accurately tracking and predicting behaviors of surrounding objects are key prerequisites for intelligent systems such as autonomous vehicles to achieve safe and high-quality decision making and motion planning. However, there still remain challenges for multi-target tracking due to object number fluctuation and occlusion. To overcome these challenges, we propose a constrained mixture sequential Monte Carlo (CMSMC) method in which a mixture representation is incorporated in the estimated posterior distribution to maintain multi-modality. Multiple targets can be tracked simultaneously within a unified framework without explicit data association between observations and tracking targets. The framework can incorporate an arbitrary prediction model as the implicit proposal distribution of the CMSMC method. An example in this paper is a learning-based model for hierarchical time-series prediction, which consists of a behavior recognition module and a state evolution module. Both modules in the proposed model are generic and flexible so as to be applied to a class of time-series prediction problems where behaviors can be separated into different levels. Finally, the proposed framework is applied to a numerical case study as well as a task of on-road vehicle tracking, behavior recognition, and prediction in highway scenarios. Instead of only focusing on forecasting trajectory of a single entity, we jointly predict continuous motions for interactive entities simultaneously. The proposed approaches are evaluated from multiple aspects, which demonstrate great potential for intelligent vehicular systems and traffic surveillance systems.
TL;DR: In this article, a new class of filtering and smoothing methods for inference in high-dimensional, nonlinear, non-Gaussian, spatio-temporal state-space models is proposed.
Abstract: We propose a new class of filtering and smoothing methods for inference in high-dimensional, nonlinear, non-Gaussian, spatio-temporal state-space models. The main idea is to combine the ensemble Ka...
17 Feb 2020
TL;DR: This letter proposes PointTrackNet, an end-to-end 3-D object detection and tracking network, to generate foreground masks,3-D bounding boxes, and point-wise tracking association displacements for each detected object.
Abstract: Recent machine learning-based multi-object tracking (MOT) frameworks are becoming popular for 3-D point clouds. Most traditional tracking approaches use filters (e.g., Kalman filter or particle filter) to predict object locations in a time sequence, however, they are vulnerable to extreme motion conditions, such as sudden braking and turning. In this letter, we propose PointTrackNet, an end-to-end 3-D object detection and tracking network, to generate foreground masks, 3-D bounding boxes, and point-wise tracking association displacements for each detected object. The network merely takes as input two adjacent point-cloud frames. Experimental results on the KITTI tracking dataset show competitive results over the state-of-the-arts, especially in the irregularly and rapidly changing scenarios.
TL;DR: This is the Editorial article summarizing the scope of the Special Issue: Approximate Bayesian Inference.
Abstract: This is the Editorial article summarizing the scope of the Special Issue: Approximate Bayesian Inference.
Proceedings Article•
01 Jan 2020TL;DR: In this article, the authors present an architecture capable of inferring an agent's goals online from both optimal and non-optimal sequences of actions, even when those actions lead to failure, enabling us to assist others when we detect that they might not achieve their goals.
Abstract: People routinely infer the goals of others by observing their actions over time. Remarkably, we can do so even when those actions lead to failure, enabling us to assist others when we detect that they might not achieve their goals. How might we endow machines with similar capabilities? Here we present an architecture capable of inferring an agent's goals online from both optimal and non-optimal sequences of actions. Our architecture models agents as boundedly-rational planners that interleave search with execution by replanning, thereby accounting for sub-optimal behavior. These models are specified as probabilistic programs, allowing us to represent and perform efficient Bayesian inference over an agent's goals and internal planning processes. To perform such inference, we develop Sequential Inverse Plan Search (SIPS), a sequential Monte Carlo algorithm that exploits the online replanning assumption of these models, limiting computation by incrementally extending inferred plans as new actions are observed. We present experiments showing that this modeling and inference architecture outperforms Bayesian inverse reinforcement learning baselines, accurately inferring goals from both optimal and non-optimal trajectories involving failure and back-tracking, while generalizing across domains with compositional structure and sparse rewards.
TL;DR: To improve the accuracy and reliability of SOH estimation and RUL prediction, a novel method based on second-order central difference particle filter (SCDPF) is proposed, by optimizing the importance probability density function, the particle degeneracy phenomenon of particlefilter (PF) can be solved.
Abstract: State of health (SOH) estimation and remaining useful life (RUL) prediction can ensure reliable and safe system operation and reduce unnecessary maintenance costs. In this paper, to improve the accuracy and reliability of SOH estimation and RUL prediction, a novel method based on second-order central difference particle filter (SCDPF) is proposed. By optimizing the importance probability density function, the particle degeneracy phenomenon of particle filter (PF) can be solved. Experiments from the National Aeronautics and Space Administration (NASA) and the Center for Advanced Life Cycle Engineering (CALCE) of the University of Maryland are conducted to demonstrate the effectiveness and satisfactory performance of the proposed SCDPF approach. The maximum error and the root mean square error (RMSE) of the SCDPF fitting approach are quite small, the minimum values of those are 0.006102 Ah and 0.001599, which are lower than those of the unscented particle filter (UPF) and particle filter (PF). The average RUL errors and average PDF width of SCDPF method are also smaller, which validates the accuracy and stability of the proposed method.
TL;DR: A new adaptive particle filter (APF) technique is proposed in this article to enhance the performance of PFs and adaptively explore the posterior space, process those low-weight particles, and tackle the sample degeneracy.
Abstract: System state estimation and prognostics are the key issues in dynamic system monitoring and management. Although the particle filter (PF) has been applied to model the nonlinear degradation feature of the system aging mechanism in several studies, it has two potential problems: the sample degeneracy and the impoverishment. To tackle these problems, a new adaptive particle filter (APF) technique is proposed in this article to enhance the performance of PFs. In the APF, a self-evaluation method is suggested to track the posterior distribution and detect the low-weight particles (sample degeneracy). A new adaptive weight adjustment approach is proposed to adaptively explore the posterior space, process those low-weight particles, and tackle the sample degeneracy. The effectiveness of the proposed APF technique is validated by simulation tests using several model conditions. It is also implemented for battery-health monitoring and prognosis. Test results show that the proposed APF technology can effectively capture and track the system’s dynamic characteristics.
TL;DR: On the basis of these metrics and visual results obtained under different environment conditions: outdoor, occluding and underwater ones, the proposed tracking scheme performs significantly better than the contemporary feature-based iterative object tracking methods and even few of the learning-based algorithms.
Abstract: In this article, a particle filter based tracking algorithm is proposed to track a target in video with vivid and complex environments. The target is represented in feature space by both color distribution and KAZE features. Color distribution is selected for its robustness to target’s scale variation and partial occlusion. KAZE features are chosen for their ability to represent the target structure and also for their superior performance in feature matching. Fusion of these two features will lead to effective tracking as compared to other features due to their better representational abilities, under challenging conditions. The trajectory of the target is established using the particle filter algorithm based on similarity between the extracted features from the target and the probable candidates in the consecutive frames. For the color distribution model, Bhattacharya coefficient is used as a similarity metric whereas Nearest Neighbor Distance Ratio is used for matching of corresponding feature points in KAZE algorithm. The particle filter update model is based on kinematic motion equations and the weights on particles are governed by an equation fusing both the color and KAZE features. Centre Location Error, Average Tracking Accuracy and Tracking Success Rate are the performance metrics considered in the evaluation process. Also, the overlap success plot and precision plot is considered for performance evaluation. On the basis of these metrics and visual results obtained under different environment conditions: outdoor, occluding and underwater ones, the proposed tracking scheme performs significantly better than the contemporary feature-based iterative object tracking methods and even few of the learning-based algorithms.
TL;DR: A particle filter-based hybrid filtering method particularly for SOC estimation of Li-ion cells in EVs and a sampling importance resampling particle filter is used in combination with a standard Kalman filter and an unscented Kalman filters as a proposal distribution for the particle filter to be made much faster and more accurate.
Abstract: Filtering based state of charge (SOC) estimation with an equivalent circuit model is commonly extended to Lithium-ion (Li-ion) batteries for electric vehicle (EV) or similar energy storage applications. During the last several decades, different implementations of online parameter identification such as Kalman filters have been presented in literature. However, if the system is a moving EV during rapid acceleration or regenerative braking or when using heating or air conditioning, most of the existing works suffer from poor prediction of state and state estimation error covariance, leading to the problem of accuracy degeneracy of the algorithm. On this account, this paper presents a particle filter-based hybrid filtering method particularly for SOC estimation of Li-ion cells in EVs. A sampling importance resampling particle filter is used in combination with a standard Kalman filter and an unscented Kalman filter as a proposal distribution for the particle filter to be made much faster and more accurate. Test results show that the error on the state estimate is less than 0.8% despite additive current measurement noise with 0.05 A deviation.
TL;DR: The computational results reveal that the proposed PLMEAPS outperforms the other two algorithms both in solutions’ quality and convergence rate when solving FJSGSP-CT.
TL;DR: A novel vessel trajectory and navigating state prediction methodology is proposed based on AIS data, which synergizes properly designed learning, motion modelling and knowledge base assisted particle filtering processes, and better prediction outperforms on account of allowing earlier alert in risk detection.
Abstract: The predictive vessel surveillance is one of the indispensable functional components in intelligent maritime traffic system. Vessel trajectory prediction serves as a prerequisite for collision detection and risk assessment. Perceiving the forthcoming traffic situation in advance helps decide the succeeding actions to mitigate the potential risk. The availability of maritime big data brings great potential to extract vessel movement patterns to support trajectory forecasting. In this paper, a novel vessel trajectory and navigating state prediction methodology is proposed based on AIS data, which synergizes properly designed learning, motion modelling and knowledge base assisted particle filtering processes. The primary contributions of this work also comprise several critical research findings to handle the key challenges in vessel trajectory and navigating state prediction problem, such as the adaptive training window determination for the learning process and effective knowledge storage and searching algorithm intended to reduce the query time of waterway pattern retrieval. The studies for these challenges are still missing in the reported literatures but they are essentially important for improving the prediction accuracy, efficiency and practicality. With the maritime traffic data collected for Singapore water, a thorough evaluation of the prediction performance has been conducted for different navigating scenarios. It is also observed that better prediction outperforms on account of allowing earlier alert in risk detection.
TL;DR: Three coordination methods are introduced based on the Infotaxis algorithm: non-coordination, passive coordination, and negotiated coordination for efficient autonomous search and estimation in a stochastic and turbulent atmospheric dispersion event.
TL;DR: Compared with other PF methods, the proposed model includes historical knowledge from similar R2F profiles, and presents good probabilistic interpretation of prediction uncertainties based on RUL distribution, and the effectiveness and superiority over other peer algorithms are justified.
TL;DR: In this article, a stochastic gradient-based particle filter (SG-PF) algorithm was proposed to estimate the unknown process outputs and parameters of an ARX model with nonlinear communication output.
Abstract: A stochastic gradient (SG)-based particle filter (SG-PF) algorithm is developed for an ARX model with nonlinear communication output in this paper. This ARX model consists of two submodels, one is a linear ARX model and the other is a nonlinear output model. The process outputs (outputs of the linear submodel) transmitted over a communication channel are unmeasurable, while the communication outputs (outputs of the nonlinear submodel) are available, and both of the two-type outputs are contaminated by white noises. Based on the rich input data and the available communication output data, a SG-PF algorithm is proposed to estimate the unknown process outputs and parameters of the ARX model. Furthermore, a direct weight optimization method and the Epanechnikov kernel method are extended to modify the particle filter when the measurement noise is a Gaussian noise with unknown variance and the measurement noise distribution is unknown. The simulation results demonstrate that the SG-PF algorithm is effective.
TL;DR: This method builds upon a number of existing algorithms in econometrics, physics, and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models.
Abstract: Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques for approximating high-dimensional probability distributions and their normalizing constants. These methods have found numerous applications in statistics and related fields; for example, for inference in nonlinear non-Gaussian state space models, and in complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which crucially impact their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. This method builds upon a number of existing algorithms in econometrics, physics and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models. We provide a theoretical analysis concerning the fluctuation and stability of this methodology that also provides insight into the properties of related algorithms. We demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications.
TL;DR: Two novel decision making methods in which reinforcement learning and particle filtering are utilized for deriving real-time maintenance policies and estimating remaining useful life for sensor-monitored degrading systems are proposed.
TL;DR: A fusion indoor positioning method that integrates the pedestrian dead-reckoning (PDR) and geomagnetic positioning by using the genetic-particle filter (GPF) algorithm that is improved compared with the PDR method, geom Magnetic positioning, and the fusion-positioning method based on the classic particle filter (PF).
Abstract: This paper proposes a fusion indoor positioning method that integrates the pedestrian dead-reckoning (PDR) and geomagnetic positioning by using the genetic-particle filter (GPF) algorithm. In the PDR module, the Mahony complementary filter (MCF) algorithm is adopted to estimate the heading angles. To improve geomagnetic positioning accuracy and geomagnetic fingerprint specificity, the geomagnetic multi-features positioning algorithm is devised and five geomagnetic features are extracted as the single-point fingerprint by transforming the magnetic field data into the geographic coordinate system (GCS). Then, an optimization mechanism is designed by using gene mutation and the method of reconstructing a particle set to ameliorate the particle degradation problem in the GPF algorithm, which is used for fusion positioning. Several experiments are conducted to evaluate the performance of the proposed methods. The experiment results show that the average positioning error of the proposed method is 1.72 m and the root mean square error (RMSE) is 1.89 m. The positioning precision and stability are improved compared with the PDR method, geomagnetic positioning, and the fusion-positioning method based on the classic particle filter (PF).
TL;DR: This paper proposes a particle centroid drift (PCD) algorithm to reduce the distance errors between nodes and obtain the particle aggregation region by using the drift vector, which has low estimation errors, a good tracking effect and an acceptable time complexity.
Abstract: The localization problem of target nodes remains unresolved, especially in large-scale and complex environments. In this paper, we propose a particle centroid drift (PCD) algorithm to reduce the distance errors between nodes and obtain the particle aggregation region by using the drift vector. First, we use the particle quality prediction function to obtain the particles in a high-likelihood region. The high-quality particles have high probability in the calculation, which can increase the number of effective particles and enable avoiding particle degradation. Then, the centroid drift vector is used to make the particle distribution similar to the actual reference distribution. Experiments are conducted on state-space models: the local movement where 55% nodes are moving and the globe movement where 100% nodes are moving. The results show that the proposed algorithm has low estimation errors, a good tracking effect and an acceptable time complexity.
Posted Content•
TL;DR: This paper develops a simple algorithm for reliably minimizing the inclusive KL, and provides a new algorithm that melds VI and MCMC, and demonstrates the utility of MSC on Bayesian probit regression for classification as well as a stochastic volatility model for financial data.
Abstract: Modern variational inference (VI) uses stochastic gradients to avoid intractable expectations, enabling large-scale probabilistic inference in complex models. VI posits a family of approximating distributions q and then finds the member of that family that is closest to the exact posterior p. Traditionally, VI algorithms minimize the "exclusive Kullback-Leibler (KL)" KL(q || p), often for computational convenience. Recent research, however, has also focused on the "inclusive KL" KL(p || q), which has good statistical properties that makes it more appropriate for certain inference problems. This paper develops a simple algorithm for reliably minimizing the inclusive KL using stochastic gradients with vanishing bias. This method, which we call Markovian score climbing (MSC), converges to a local optimum of the inclusive KL. It does not suffer from the systematic errors inherent in existing methods, such as Reweighted Wake-Sleep and Neural Adaptive Sequential Monte Carlo, which lead to bias in their final estimates. We illustrate convergence on a toy model and demonstrate the utility of MSC on Bayesian probit regression for classification as well as a stochastic volatility model for financial data.
07 Jun 2020
TL;DR: A Reinforcement Learning (RL)-based approach is proposed that can achieve the URLLC requirements in a typical intersection scenario, and results show that enhanced EKF and PF methods achieve packet delay more than 10 ms, whereas the proposed deep RL-based method can reduce the latency to about 6 ms, by extracting context information from the training data.
Abstract: Ultra-Reliable and Low-Latency Communications (URLLC) services in vehicular networks on millimeter-wave bands present a significant challenge, considering the necessity of constantly adjusting the beam directions. Conventional methods are mostly based on classical control theory, e.g., Kalman filter and its variations, which mainly deal with stationary scenarios. Therefore, severe application limitations exist, especially with complicated, dynamic Vehicle-to-Everything (V2X) channels. This paper gives a thorough study of this subject, by first modifying the classical approaches, e.g., Extended Kalman Filter (EKF) and Particle Filter (PF), for non-stationary scenarios, and then proposing a Reinforcement Learning (RL)-based approach that can achieve the URLLC requirements in a typical intersection scenario. Simulation results based on a commercial ray-tracing simulator show that enhanced EKF and PF methods achieve packet delay more than 10 ms, whereas the proposed deep RL-based method can reduce the latency to about 6 ms, by extracting context information from the training data.