Journal ArticleDOI
Audio–Visual Particle Flow SMC-PHD Filtering for Multi-Speaker Tracking
Reads0
Chats0
TLDR
This work proposes a new framework where particle flow (PF) is used to migrate particles smoothly from the prior to the posterior probability density, and developed two new algorithms, AV-ZPF-SMC-PHD and AV-NPF-S MC-P HD, where the speaker states from the previous frames are also considered for particle relocation.Abstract:
Sequential Monte Carlo probability hypothesis density (SMC-PHD) filtering is a popular method used recently for audio-visual (AV) multi-speaker tracking. However, due to the weight degeneracy problem, the posterior distribution can be represented poorly by the estimated probability, when only a few particles are present around the peak of the likelihood density function. To address this issue, we propose a new framework where particle flow (PF) is used to migrate particles smoothly from the prior to the posterior probability density. We consider both zero and non-zero diffusion particle flows (ZPF/NPF), and developed two new algorithms, AV-ZPF-SMC-PHD and AV-NPF-SMC-PHD, where the speaker states from the previous frames are also considered for particle relocation. The proposed algorithms are compared systematically with several baseline tracking methods using the AV16.3, AVDIAR and CLEAR datasets, and are shown to offer improved tracking accuracy and average effective sample size (ESS).read more
Citations
More filters
Posted Content
End-To-End Semi-supervised Learning for Differentiable Particle Filters
TL;DR: An end-to-end learning objective is presented based upon the maximisation of a pseudo-likelihood function which can improve the estimation of states when large portion of true states are unknown and is assessed in state estimation tasks in robotics with simulated and real-world datasets.
Dataset
Data and Codes for reproducing the results in "Mean-Shift and Sparse Sampling Based SMC-PHD Filtering for Audio Informed Visual Speaker Tracking"
TL;DR: The audio data is proposed to be used to improve the visual SMC-PHD (V-SMC-P HD) filter by using the direction of arrival angles of the audio sources to determine when to propagate the born particles and reallocate the surviving and spawned particles.
Proceedings ArticleDOI
End-to-End Semi-supervised Learning for Differentiable Particle Filters
TL;DR: In this paper, an end-to-end learning objective based on the maximisation of a pseudo-likelihood function is proposed to improve the estimation of states when large portion of true states are unknown.
Journal ArticleDOI
Audio-Visual Event Localization by Learning Spatial and Semantic Co-Attention
TL;DR: In this paper , a co-attention model is proposed to exploit the spatial and semantic correlations between the audio and visual features, which helps guide the extraction of discriminative features for better event localization.
Journal ArticleDOI
Joint detection and tracking of non-ellipsoidal extended targets based on cubature Kalman-CBMeMBer sub-random matrices filter
TL;DR: In this article, the authors proposed a new approach for the ESTs tracking under the non-linear Gaussian system based on track-before-detect (TBD) approach, which is more accurate and more principled in mathematical terms compared to SMC-CBMeMBer filter.
References
More filters
Journal ArticleDOI
On spatial smoothing for direction-of-arrival estimation of coherent signals
TL;DR: An analysis of a "spatial smoothing" preprocessing scheme, recently suggested by Evans et al., to circumvent problems encountered in direction-of-arrival estimation of fully correlated signals.
Proceedings Article
The Unscented Particle Filter
TL;DR: This paper proposes a new particle filter based on sequential importance sampling that outperforms standard particle filtering and other nonlinear filtering methods very substantially and is in agreement with the theoretical convergence proof for the algorithm.
Journal ArticleDOI
Sequential Monte Carlo methods for multitarget filtering with random finite sets
TL;DR: In this paper, a sequential Monte Carlo (SMC) multitarget filter is proposed and demonstrated on a number of simulated scenarios, which is suitable for problems involving nonlinear nonGaussian dynamics.
Book ChapterDOI
A Boosted Particle Filter: Multitarget Detection and Tracking
TL;DR: This work introduces a vision system that is capable of learning, detecting and tracking the objects of interest, and interleaving Adaboost with mixture particle filters, a simple, yet powerful and fully automatic multiple object tracking system.
Journal Article
Bayesian statistics without tears: A sampling-resampling perspective
TL;DR: In this article, a sampling-resampling perspective on Bayesian inference is presented, which has both pedagogic appeal and suggests easily implemented calculation strategies, such as sampling-based methods.
Related Papers (5)
Particle flow for sequential Monte Carlo implementation of probability hypothesis density
Yang Liu,Wenwu Wang,Yuxin Zhao +2 more