There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Monthly Notices is one of the three largest general primary astronomical research publications. It is an international journal, published by the Royal Astronomical Society. This article 1 describes its publication policy and practice.

Monthly Notices of the Royal Astronomical Society

Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these developments have always been to the benefit of image interpretation and machine vision, only recently has it become evident that machine learning architectures, and deep neural networks in particular, can be effective for computational image formation, aside from interpretation. The deep learning approach has proven to be especially attractive when the measurement is noisy and the measurement operator ill posed or uncertain. Examples reviewed here are: super-resolution; lensless retrieval of phase and complex amplitude from intensity; photon-limited scenes, including ghost imaging; and imaging through scatter. In this paper, we cast these works in a common framework. We relate the deep-learning-inspired solutions to the original computational imaging formulation and use the relationship to derive design insights, principles, and caveats of more general applicability. We also explore how the machine learning process is aided by the physics of imaging when ill posedness and uncertainties become particularly severe. It is hoped that the present unifying exposition will stimulate further progress in this promising field of research.

On the use of deep learning for computational imaging

How to image objects that are hidden from a camera's view is a problem of fundamental importance to many fields of research, with applications in robotic vision, defence, remote sensing, medical imaging and autonomous vehicles. Non-line-of-sight (NLOS) imaging at macroscopic scales has been demonstrated by scanning a visible surface with a pulsed laser and a time-resolved detector. Whereas light detection and ranging (LIDAR) systems use such measurements to recover the shape of visible objects from direct reflections, NLOS imaging reconstructs the shape and albedo of hidden objects from multiply scattered light. Despite recent advances, NLOS imaging has remained impractical owing to the prohibitive memory and processing requirements of existing reconstruction algorithms, and the extremely weak signal of multiply scattered light. Here we show that a confocal scanning procedure can address these challenges by facilitating the derivation of the light-cone transform to solve the NLOS reconstruction problem. This method requires much smaller computational and memory resources than previous reconstruction methods do and images hidden objects at unprecedented resolution. Confocal scanning also provides a sizeable increase in signal and range when imaging retroreflective objects. We quantify the resolution bounds of NLOS imaging, demonstrate its potential for real-time tracking and derive efficient algorithms that incorporate image priors and a physically accurate noise model. Additionally, we describe successful outdoor experiments of NLOS imaging under indirect sunlight.

Confocal non-line-of-sight imaging based on the light-cone transform.

A three-photon entangled state with 3 × 3 × 2 dimensions of its orbital angular momentum is created by using two independent entangled photon pairs from two nonlinear crystals, enabling the development of a new layered quantum communication protocol. Forming the backbone of quantum technologies today, entanglement1,2 has been demonstrated in physical systems as diverse as photons3, ions4 and superconducting circuits5. Although steadily pushing the boundary of the number of particles entangled, these experiments have remained in a two-dimensional space for each particle. Here we show the experimental generation of the first multi-photon entangled state where both the number of particles and dimensions are greater than two. Two photons in our state reside in a three-dimensional space, whereas the third lives in two dimensions. This asymmetric entanglement structure6 only appears in multiparticle entangled states with d > 26. Our method relies on combining two pairs of photons, high-dimensionally entangled in their orbital angular momentum7. In addition, we show how this state enables a new type of ‘layered’ quantum communication protocol. Entangled states such as these serve as a manifestation of the complex dance of correlations that can exist within quantum mechanics.

Multi-photon entanglement in high dimensions

Scientists demonstrate an experimental method that allows them to locate and track moving targets that are hidden from the direct line of sight, for example, by a wall or an obstacle, with only a few seconds acquisition time and centimetre precision. The ability to detect motion and track a moving object hidden around a corner or behind a wall provides a crucial advantage when physically going around the obstacle is impossible or dangerous. Previous methods have demonstrated that it is possible to reconstruct the shape of an object hidden from view. However, these methods do not enable the tracking of movement in real time. We demonstrate a compact non-line-of-sight laser ranging technology that relies on the ability to send light around an obstacle using a scattering floor and then detect the return signal from a hidden object within only a few seconds of acquisition time. By detecting this signal with a single-photon avalanche diode (SPAD) camera, we follow the movement of an object located a metre away from the camera with centimetre precision. We discuss the possibility of applying this technology to a variety of real-life situations in the near future.

/pdf/detection-and-tracking-of-moving-objects-hidden-from-view-2rorllupvx.pdf

Detection and tracking of moving objects hidden from view

We consider the problem of wireless communication across medium boundaries, specifically across the water-air interface. In particular, we are interested in enabling a submerged underwater sensor to directly communicate with an airborne node. Today's communication technologies cannot enable such a communication link. This is because no single type of wireless signal can operate well across different media and most wireless signals reflect back at media boundaries. We present a new communication technology, translational acoustic-RF communication (TARF). TARF enables underwater nodes to directly communicate with airborne nodes by transmitting standard acoustic signals. TARF exploits the fact that underwater acoustic signals travel as pressure waves, and that these waves cause displacements of the water surface when they impinge on the water-air boundary. To decode the transmitted signals, TARF leverages an airborne radar which measures and decodes these surface displacements. We built a prototype of TARF that incorporates algorithms for dealing with the constraints of this new communication modality. We evaluated TARF in controlled and uncontrolled environments and demonstrated that it enables the first practical communication link across the water-air interface. Our results show that TARF can achieve standard underwater bitrates up to 400bps, and that it can operate correctly in the presence of surface waves with amplitudes up to 16 cm peak-to-peak, i.e., 100,000X larger than the surface perturbations caused by TARF's underwater acoustic transmitter.

Networking across boundaries: enabling wireless communication through the water-air interface

Gravitational wave (GW) detection is now commonplace and as the sensitivity of the global network of GW detectors improves, we will observe $\mathcal{O}(100)$s of transient GW events per year. The current methods used to estimate their source parameters employ optimally sensitive but computationally costly Bayesian inference approaches where typical analyses have taken between 6 hours and 5 days. For binary neutron star and neutron star black hole systems prompt counterpart electromagnetic (EM) signatures are expected on timescales of 1 second -- 1 minute and the current fastest method for alerting EM follow-up observers, can provide estimates in $\mathcal{O}(1)$ minute, on a limited range of key source parameters. Here we show that a conditional variational autoencoder pre-trained on binary black hole signals can return Bayesian posterior probability estimates. The training procedure need only be performed once for a given prior parameter space and the resulting trained machine can then generate samples describing the posterior distribution $\sim 6$ orders of magnitude faster than existing techniques.

Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy.

Imaging through a strongly diffusive medium remains an outstanding challenge, in particular in applications in biological and medical imaging. Here, we propose a method based on a single-photon time-of-flight camera that allows, in combination with computational processing of the spatial and full temporal photon distribution data, imaging of an object embedded inside a strongly diffusive medium over more than 80 transport mean free paths. The technique is contactless and requires 1 s acquisition times, thus allowing Hz frame rate imaging. The imaging depth corresponds to several centimetres of human tissue and allows us to perform deep-body imaging as a proof of principle. By combining a single-photon time-of-flight camera with computational processing of the spatial and full temporal photon distribution data, an object embedded inside a strongly diffusive medium can be imaged over more than 80 transport mean free paths in a contactless manner on the timescale of the order of 1 s.

/pdf/computational-time-of-flight-diffuse-optical-tomography-2t0v4q7gm0.pdf

Computational time-of-flight diffuse optical tomography

We use a femtosecond laser and a single-photon avalanche diode (SPAD) camera to develop a system that is able to locate objects hidden from view and track their motion.

/pdf/detection-and-tracking-of-moving-objects-hidden-from-view-ewkohc3qeq.pdf

Francesco Tonolini

Papers

Detection and tracking of moving objects hidden from view

Networking across boundaries: enabling wireless communication through the water-air interface

Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy.

Computational time-of-flight diffuse optical tomography

Detection and tracking of moving objects hidden from view