Showing papers by "Stephen McLaughlin published in 2019"

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers.

Abstract: Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications. The use of single-photon data has been limited by time-consuming reconstruction algorithms. Here, the authors combine statistical models and computational tools known from computer graphics and show real-time reconstruction of moving scenes.

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

Abstract: Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications.

Bayesian 3D Reconstruction of Complex Scenes from Single-Photon Lidar Data

Abstract: Light detection and ranging (Lidar) data can be used to capture the depth and intensity profile of a 3D scene. This modality relies on constructing, for each pixel, a histogram of time delays between emitted light pulses and detected photon arrivals. In a general setting, more than one surface can be observed in a single pixel. The problem of estimating the number of surfaces, their reflectivity, and position becomes very challenging in the low-photon regime (which equates to short acquisition times) or relatively high background levels (i.e., strong ambient illumination). This paper presents a new approach to 3D reconstruction using single-photon, single-wavelength Lidar data, which is capable of identifying multiple surfaces in each pixel. Adopting a Bayesian approach, the 3D structure to be recovered is modelled as a marked point process, and reversible jump Markov chain Monte Carlo (RJ-MCMC) moves are proposed to sample the posterior distribution of interest. In order to promote spatial correlation between points belonging to the same surface, we propose a prior that combines an area interaction process and a Strauss process. New RJ-MCMC dilation and erosion updates are presented to achieve an efficient exploration of the configuration space. To further reduce the computational load, we adopt a multiresolution approach, processing the data from a coarse to the finest scale. The experiments performed with synthetic and real data show that the algorithm obtains better reconstructions than other recently published optimization algorithms for lower execution times.

3D Reconstruction Using Single-photon Lidar Data Exploiting the Widths of the Returns

Abstract: Single-photon light detection and ranging (Lidar) data can be used to capture depth and intensity profiles of a 3D scene. In a general setting, the scenes can have an unknown number of surfaces per pixel (semi-transparent surfaces or outdoor measurements), high background noise (strong ambient illumination), can be acquired by systems with a broad instrumental response (non-parallel laser beam with respect to the target surface) and with possibly high attenuating media (underwater conditions). The existing methods generally tackle only a subset of these problems and can fail in a more general scenario. In this paper, we propose a new 3D reconstruction algorithm that can handle all the aforementioned difficulties. The novel algorithm estimates the broadening of the impulse response, considers the attenuation induced by scattering media, while allowing for multiple surfaces per pixel. A series of experiments performed in real long-range and underwater Lidar datasets demonstrate the performance of the proposed method.

Fast online 3D reconstruction of dynamic scenes from individual single-photon detection events

Abstract: In this paper, we present an algorithm for online 3D reconstruction of dynamic scenes using individual times of arrival (ToA) of photons recorded by single-photon detector arrays. One of the main challenges in 3D imaging using single-photon Lidar is the integration time required to build ToA histograms and reconstruct reliable 3D profiles in the presence of non-negligible ambient illumination. This long integration time also prevents the analysis of rapid dynamic scenes using existing techniques. We propose a new method which does not rely on the construction of ToA histograms but allows, for the first time, individual detection events to be processed online, in a parallel manner in different pixels, while accounting for the intrinsic spatiotemporal structure of dynamic scenes. Adopting a Bayesian approach, a Bayesian model is constructed to capture the dynamics of the 3D profile and an approximate inference scheme based on assumed density filtering is proposed, yielding a fast and robust reconstruction algorithm able to process efficiently thousands to millions of frames, as usually recorded using single-photon detectors. The performance of the proposed method, able to process hundreds of frames per second, is assessed using a series of experiments conducted with static and dynamic 3D scenes and the results obtained pave the way to a new family of real-time 3D reconstruction solutions.

Fast Adaptive Scene Sampling for Single-Photon 3D Lidar Images

Abstract: Reducing acquisition time is a major challenge for single-photon based imaging. This paper presents a new approach for adaptive scene sampling allowing for faster acquisition when compared to classical uniform sampling or random sampling strategies. The approach is applied to the laser detection and ranging (Lidar) three-dimensional (3D) imaging where sampling is optimized regarding the depth image. Based on data statistics, the approach starts by achieving a robust estimation of the depth image. The latter is used to generate a map of regions of interest that informs next samples positions and their acquisition times. The process is repeated until a stopping criterion is met. A particular interest is given to fast processing to allow real-world application of the proposed approach. Results on real data show the benefits of this strategy that can reduce acquisition times by a factor of 8 compared to uniform sampling in some scenarios.

Fast Surface Detection in Single-Photon Lidar Waveforms

Abstract: Single-photon light detection and ranging (Lidar) devices can be used to obtain range and reflectivity information from 3D scenes. However, reconstructing the 3D surfaces from the raw waveforms can be very challenging, in particular when the number of spurious background detections is large compared to the number of signal detections. This paper introduces a new and fast detection algorithm, which can be used to assess the presence of objects/surfaces in each waveform, allowing only the histograms where the imaged surfaces are present to be further processed. The method is compared to state-of-the-art 3D reconstruction methods using synthetic and real single-photon data and the results illustrate its benefits for fast and robust target detection using single-photon data.

Bayesian 3D Reconstruction of Subsampled Multispectral Single-photon Lidar Signals

Abstract: Light detection and ranging (Lidar) single-photon devices capture range and intensity information from a 3D scene. This modality enables long range 3D reconstruction with high range precision and low laser power. A multispectral single-photon Lidar system provides additional spectral diversity, allowing the discrimination of different materials. However, the main drawback of such systems can be the long acquisition time needed to collect enough photons in each spectral band. In this work, we tackle this problem in two ways: first, we propose a Bayesian 3D reconstruction algorithm that is able to find multiple surfaces per pixel, using few photons, i.e., shorter acquisitions. In contrast to previous algorithms, the novel method processes the jointly all the spectral bands, obtaining better reconstructions using less photon detections. The proposed model promotes spatial correlation between neighbouring points within a given surface using spatial point processes. Secondly, we account for different spatial and spectral subsampling schemes, which reduce the total number of measurements, without significant degradation of the reconstruction performance. In this way, the total acquisition time, memory requirements and computational time can be significantly reduced. The experiments performed using both synthetic and real single-photon Lidar data demonstrate the advantages of tailored sampling schemes over random alternatives. Furthermore, the proposed algorithm yields better estimates than other existing methods for multi-surface reconstruction using multispectral Lidar data.

A Hierarchical Bayesian Approach to Neutron Spectrum Unfolding With Organic Scintillators

Abstract: We propose a hierarchical Bayesian model and a state-of-the-art Monte Carlo sampling method to solve the unfolding problem, i.e., to estimate the spectrum of an unknown neutron source from the data detected by an organic scintillator. Inferring neutron spectra is important for several applications, including nonproliferation and nuclear security, as it allows the discrimination of fission sources in special nuclear material (SNM) from other types of neutron sources based on the differences of the emitted neutron spectra. Organic scintillators interact with neutrons mostly via elastic scattering on hydrogen nuclei and therefore partially retain neutron energy information. Consequently, the neutron spectrum can be derived through deconvolution of the measured light-output spectrum and the response functions of the scintillator to monoenergetic neutrons. The proposed approach is compared to three existing methods using the simulated data to enable controlled benchmarks. We consider three sets of detector responses. One set corresponds to a 2.5-MeV monoenergetic neutron source and two sets are associated with (energywise) continuous neutron sources (252Cf and 241AmBe). Our results show that the proposed method has similar or better unfolding performance compared with other iterative or Tikhonov regularization-based approaches in terms of accuracy and robustness against limited detection events while requiring less user supervision. The proposed method also provides a posteriori confidence measures, which offers additional information regarding the uncertainty of the measurements and the extracted information.

On fast object detection using single-photon lidar data

Abstract: Light detection and ranging (Lidar) systems based on single-photon detection can be used to obtain range and reflectivity information from 3D scenes with high range resolution. However, reconstructing the 3D surfaces from the raw single-photon waveforms is challenging, in particular when a limited number of photons is detected and when the ratio of spurious background detection events is large. This paper reviews a set of fast detection algorithms, which can be used to assess the presence of objects/surfaces in each waveform, allowing only the histograms where the imaged surfaces are present to be further processed. The original method we recently proposed is extended here using a multiscale approach to further reduce the computational complexity of the detection process. The proposed methods are compared to state-of-the-art 3D reconstruction methods using synthetic and real single-photon data and the results illustrate their benefits for fast and robust target detection.

ToCo: An Ontology for Representing Hybrid Telecommunication Networks

Abstract: The TOUCAN project proposed an ontology for telecommunication networks with hybrid technologies – the TOUCAN Ontology (ToCo), available at http://purl.org/toco/, as well as a knowledge design pattern Device-Interface-Link (DIL) pattern. The core classes and relationships forming the ontology are discussed in detail. The ToCo ontology can describe the physical infrastructure, quality of channel, services and users in heterogeneous telecommunication networks which span multiple technology domains. The DIL pattern is observed and summarised when modelling networks with various technology domains. Examples and use cases of ToCo are presented for demonstration.

Real-time 3D reconstruction of complex scenes using single-photon lidar: when image processing meets computer graphics

Abstract: Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications.

Characterising cross-coupling in coherent fibre bundles

Abstract: Fibre-bundle endomicroscopy is an emerging medical imaging tool. Inter-core coupling within coherent fibre bundles limits the technology's imaging capabilities. We introduce a novel approach for quantifying and modelling cross coupling, optimising image reconstruction.

SARA – A Semantic Access Point Resource Allocation Service for Heterogenous Wireless Networks

Abstract: In this paper, we present SARA, a Semantic Access point Resource Allocation service for heterogenous wireless networks with various wireless access technologies existing together. By automatically reasoning on the knowledge base of the full system provided by a knowledge based autonomic network management system – SEANET, SARA selects the access point providing the best quality of service among the different access technologies. Based on an ontology assisted knowledge based system SEANET, SARA can also adapt the access point selection strategy according to customer defined rules automatically. Results of our evaluation based on emulated networks with hybrid access technologies and various scales show that SARA is able to improve the channel condition, in terms of throughput, evidently. Comparisons with current AP selection algorithms demonstrate that SARA outperforms the existing AP selection algorithms. The overhead in terms of time expense is reasonable and is shown to be faster than traditional access point selection approaches.

Segmentation And Quantification Of Cellular Load In Pulmonary Endomicroscopic Images Using Convolutional Neural Networks

Abstract: Fibre Bundle Endomicroscopy (FBEμ) is an emerging tool that facilitates the real-time structural and functional (via fluorescent dyes) imaging of the distal lung, providing valuable in vivo, in situ indicators across a range pathological or physiological processes. This paper proposes a novel approach for localising and quantifying abnormalities in distal lung, such as increased cellular load, through semantic image segmentation. Two Convolutional Neural Network (CNN) architectures have been tested, (i) U-Net, a purpose specific network for biomedical image applications, and (ii) ENet, a network optimised for fast inference. The results indicate that semantic segmentation of cells as well as quantification of cellular load is viable, with U-Net consistently outperforming ENet, obtaining a pixel accuracy of 0.842 and a correlation (r) with the corresponding manual cellular load estimation of 0. 866.*

EM-based approach to 3D reconstruction from single-waveform multispectral Lidar data

Abstract: In this paper, we present a novel Bayesian approach for estimating spectral and range profiles from single-photon Lidar waveforms associated with single surfaces in the photon-limited regime. In contrast to classical multispectral Lidar signals, we consider a single Lidar waveform per pixel, whereby a single detector is used to acquire information simultaneously at multiple wavelengths. A new observation model based on a mixture of distributions is developed. It relates the unknown parameters of interest to the observed waveforms containing information from multiple wavelengths. Adopting a Bayesian approach, several prior models are investigated and a stochastic Expectation-Maximization algorithm is proposed to estimate the spectral and depth profiles. The reconstruction performance and computational complexity of our approach are assessed, for different prior models, through a series of experiments using synthetic and real data under different observation scenarios. The results obtained demonstrate a significant speed-up without significant degradation of the reconstruction performance when compared to existing methods in the photon-starved regime.

Optimising DVB-S2(X) Links in Nonlinear Environments

Abstract: Satellite communication standards such as the second-generation digital video broadcasting for satellites, i.e., DVB-S2(X), exploit phase-shift keying (PSK) and amplitude and phase-shift keying (APSK) modulation schemes as they are more robust against nonlinearity than quadrature amplitude modulations (QAMs) [1] [2]. However, the system performance of satellite communication remains profoundly influenced by the presence of the nonlinear high power amplifiers (HPA). In this contribution, we analytically and numerically study the end-toend distortion of the APSK constellation from the digital IQ signal at the transmitter across to the nonlinear channel and back to the recovered digital IQ signal at the receiver. We demonstrate that the distortion depends not only on the input back-off (IBO) of the HPA but, critically, also on the roll-off of the matched digital root raised cosine (RRC) bandwidth-limiting filters used at transmitters and receivers. Consequently, we derive informed guidelines for the adaptive operation of satellite links. Focusing on 16 APSK (4+12 version) modulation, the main contributions of our work include; 1. We present an accurate estimation of the Gaussian spreading of the received symbols around reference constellation points. The digital pre-distortion at the transmitter is applied. It has been shown that the spreading depends on the choice of RRC roll-off factor and HPA IBO. This estimation enables us to obtain the equivalent additive Gaussian noise determined by the combined RRC and HPA characteristics. Significantly, we then proceed to demonstrate that the performance of the nonlinear channel can be approximated by a linear channel with an additional known noise source. 2. The informed estimation of the optimum IBO which can minimise the system bit error rate (BER) is obtained. Importantly we show that the optimum IBO depends on both the channel noise and the roll-off. 3. An adaptive operation of DVB-S2X link with dynamic power control is achieved, which maintains performance close to optimum by computationally efficient estimation of the combination of MODCOD, RRC roll-off and IBO operation on a given HPA. The paper is organized as follows. In Section II, we provide some backgrounds including the satellite link descriptions, the adverse impacts from nonlinearity and inter-symbol interference (ISI). In Section III, we introduce the digital pre-distortion algorithm and obtain the equivalent linear link by ‘translating’ the ISI into equivalent Gaussian noise. Then, the optimum HPA IBO under different RRC roll-off and channel noise is achieved. In Section IV, an adaptive operation specifying the combination of MODCOD, RRC roll-off and HPA IBO is proposed, which can maximise the system spectrum efficiency while still guaranteeing the BER requirement. Conclusions are drawn in Section V.

Joint Reconstruction of Multitemporal or Multispectral Single-Photon 3D LiDAR Images

Abstract: The aim of this paper is to propose a specialized algorithm to process Multitemporal or Multispectral 3D single-photon Lidar images. Of particular interest are challenging scenarios often encountered in real world, i.e., imaging through obscurants such as water, fog or imaging multilayered targets such as target behind camouflage. To restore the data, the algorithm accounts for data Poisson statistics and available prior knowledge regarding target depth and reflectivity estimates. More precisely, it accounts for (a) the non-local spatial correlations between pixels, (b) the spatial clustering of target returned photons and (c) spectral and temporal correlations between frames. An alternating direction method of multipliers (ADMM) algorithm is used to minimize the resulting cost function since it offers good convergence properties. The algorithm is validated on real data which show the benefit of the proposed strategy especially when dealing with multi-dimensional 3D data.

Lost Silence: An emergency response early detection service through continuous processing of telecommunication data streams

Abstract: Early detection of significant traumatic events, e.g. a terrorist attack or a ship capsizing, is important to ensure that a prompt emergency response can occur. In the modern world telecommunication systems could play a key role in ensuring a successful emergency response by detecting such incidents through significant changes in calls and access to the networks. In this paper a methodology is illustrated to detect such incidents immediately (with the delay in the order of milliseconds), by processing semantically annotated streams of data in cellular telecommunication systems. In our methodology, live information about the position and status of phones are encoded as RDF streams. We propose an algorithm that processes streams of RDF annotated telecommunication data to detect abnormality. Our approach is exemplified in the context of a passenger cruise ship capsizing. However, the approach is readily translatable to other incidents. Our evaluation results show that with a properly chosen window size, such incidents can be detected efficiently and effectively.

Real-Time 3D Color Imaging with Single-Photon Lidar Data

Abstract: Single-photon lidar devices can acquire 3D data at very long range with high precision. Moreover, recent advances in lidar arrays have enabled acquisitions at very high frame rates. However, these devices place a severe bottleneck on the reconstruction algorithms, which have to handle very large volumes of noisy data. Recently, real-time 3D reconstruction of distributed surfaces has been demonstrated obtaining information at one wavelength. Here, we propose a new algorithm that achieves color 3D reconstruction without increasing the execution time nor the acquisition process of the realtime single-wavelength reconstruction system. The algorithm uses a coded aperture that compresses the data by considering a subset of the wavelengths per pixel. The reconstruction algorithm is based on a plug-and-play denoising framework, which benefits from off-the-shelf point cloud and image de-noisers. Experiments using real lidar data show the competitivity of the proposed method.

Patch-Based Sparse Representation For Bacterial Detection

Abstract: In this paper, we propose an unsupervised approach for bacterial detection in optical endomicroscopy images. This approach splits each image into a set of overlapping patches and assumes that observed intensities are linear combinations of the actual intensity values associated with background image structures, corrupted by additive Gaussian noise and potentially by a sparse outlier term modelling anomalies (which are considered to be candidate bacteria). The actual intensity term representing background structures is modelled as a linear combination of a few atoms drawn from a dictionary which is learned from bacteria-free data and then fixed while analyzing new images. The bacteria detection task is formulated as a minimization problem and an alternating direction method of multipliers (ADMM) is then used to estimate the unknown parameters. Simulations conducted using two ex vivo lung datasets show good detection and correlation performance between bacteria counts identified by a trained clinician and those of the proposed method.

Underwater depth imaging using time-correlated single-photon counting at video frame rates

Abstract: This paper presents the Time-Correlated Single-Photon Counting (TCSPC) technique applied to underwater environments in order to reconstruct three-dimensional scenes. Two different transceiver systems approaches are described. The first transceiver comprised a single-pixel monostatic scanning unit, which used a fiber-coupled silicon single-photon avalanche diode (SPAD) detector, and a fiber-coupled supercontinuum laser source used in conjunction with an acousto-optic tunable filter (AOTF) for wavelength selection. The experiments were performed using the supercontinuum pulsed laser source operating at a repetition rate of 19.5 MHz, fiber coupled to the AOTF in order to select one operational wavelength, tuned for best performance for the level of scattering of the particular underwater environment. Laboratory-based experiments were performed using average optical powers of less than 1 mW and depth profiles were acquired at up to 8 attenuation lengths between the transceiver and target. The second transceiver system was based on a complementary metal-oxide semiconductor (CMOS) SPAD detector array in a bistatic configuration. It comprised an array of 192 × 128 SPAD detectors, with each detector having an integrated time to digital converter, and a laser diode operating at a wavelength of 670 nm, a repetition rate of 40 MHz, and average optical power up to 9 mW. The experiments demonstrated the recovery of intensity and depth profiles associated with moving targets at up to 4 attenuation lengths. Using data from both systems, various image processing techniques were investigated to reconstruct target depth and intensity profiles in highly scattering underwater environments.

Spectrum-Energy-Economy Efficiency Trade-off of Wireless Communication Systems with Separated Indoor/Outdoor Scenarios for 5G and B5G

Abstract: In this paper, we consider a heterogeneous 5G cellular architecture that separates the outdoor and indoor scenarios and in particular study the trade-off between the spectrum efficiency (SE), energy efficiency (EE), economy efficiency (ECE). Mathematical expressions for the system capacity, EE, SE, and ECE respectively are derived using a proposed realistic power consumption model. The comparison of system performance in terms of SE, EE, and ECE, results in the observation that the proposed network architecture, which separates the outdoor and indoor scenarios, offers a promising solution for future communication systems that have strict requirements on the data rate and efficiency.