Machine learning (ML) encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent years. This article reviews in a selective way the recent research on the interface between machine learning and the physical sciences. This includes conceptual developments in ML motivated by physical insights, applications of machine learning techniques to several domains in physics, and cross fertilization between the two fields. After giving a basic notion of machine learning methods and principles, examples are described of how statistical physics is used to understand methods in ML. This review then describes applications of ML methods in particle physics and cosmology, quantum many-body physics, quantum computing, and chemical and material physics. Research and development into novel computing architectures aimed at accelerating ML are also highlighted. Each of the sections describe recent successes as well as domain-specific methodology and challenges.

Machine learning and the physical sciences

The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques, in order to help manage the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space. In this paper, we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-the-art in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.

/pdf/deep-learning-in-mobile-and-wireless-networking-a-survey-ct8cy2kuqi.pdf

Deep Learning in Mobile and Wireless Networking: A Survey

Quantitative phase imaging (QPI) has emerged as a valuable method for investigating cells and tissues. QPI operates on unlabelled specimens and, as such, is complementary to established fluorescence microscopy, exhibiting lower phototoxicity and no photobleaching. As the images represent quantitative maps of optical path length delays introduced by the specimen, QPI provides an objective measure of morphology and dynamics, free of variability due to contrast agents. Owing to the tremendous progress witnessed especially in the past 10–15 years, a number of technologies have become sufficiently reliable and translated to biomedical laboratories. Commercialization efforts are under way and, as a result, the QPI field is now transitioning from a technology-development-driven to an application-focused field. In this Review, we aim to provide a critical and objective overview of this dynamic research field by presenting the scientific context, main principles of operation and current biomedical applications. Over the past 10–15 years, quantitative phase imaging has moved from a research-driven to an application-focused field. This Review presents the main principles of operation and representative basic and clinical science applications.

Quantitative phase imaging in biomedicine

We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this framework, we improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching the resolution that is acquired using high-numerical-aperture objectives. We also demonstrate cross-modality super-resolution, transforming confocal microscopy images to match the resolution acquired with a stimulated emission depletion (STED) microscope. We further demonstrate that total internal reflection fluorescence (TIRF) microscopy images of subcellular structures within cells and tissues can be transformed to match the results obtained with a TIRF-based structured illumination microscope. The deep network rapidly outputs these super-resolved images, without any iterations or parameter search, and could serve to democratize super-resolution imaging.

Deep learning enables cross-modality super-resolution in fluorescence microscopy

Research in photonic computing has flourished due to the proliferation of optoelectronic components on photonic integration platforms. Photonic integrated circuits have enabled ultrafast artificial neural networks, providing a framework for a new class of information processing machines. Algorithms running on such hardware have the potential to address the growing demand for machine learning and artificial intelligence in areas such as medical diagnosis, telecommunications, and high-performance and scientific computing. In parallel, the development of neuromorphic electronics has highlighted challenges in that domain, particularly related to processor latency. Neuromorphic photonics offers sub-nanosecond latencies, providing a complementary opportunity to extend the domain of artificial intelligence. Here, we review recent advances in integrated photonic neuromorphic systems, discuss current and future challenges, and outline the advances in science and technology needed to meet those challenges. Photonics offers an attractive platform for implementing neuromorphic computing due to its low latency, multiplexing capabilities and integrated on-chip technology.

Photonics for artificial intelligence and neuromorphic computing

Deep learning has been transforming our ability to execute advanced inference tasks using computers. Here we introduce a physical mechanism to perform machine learning by demonstrating an all-optical diffractive deep neural network (D2NN) architecture that can implement various functions following the deep learning-based design of passive diffractive layers that work collectively. We created 3D-printed D2NNs that implement classification of images of handwritten digits and fashion products, as well as the function of an imaging lens at a terahertz spectrum. Our all-optical deep learning framework can perform, at the speed of light, various complex functions that computer-based neural networks can execute; will find applications in all-optical image analysis, feature detection, and object classification; and will also enable new camera designs and optical components that perform distinctive tasks using D2NNs.

/pdf/all-optical-machine-learning-using-diffractive-deep-neural-1o1xdg7mzl.pdf

All-optical machine learning using diffractive deep neural networks

We introduce an all-optical Diffractive Deep Neural Network (D2NN) architecture that can learn to implement various functions after deep learning-based design of passive diffractive layers that work collectively. We experimentally demonstrated the success of this framework by creating 3D-printed D2NNs that learned to implement handwritten digit classification and the function of an imaging lens at terahertz spectrum. With the existing plethora of 3D-printing and other lithographic fabrication methods as well as spatial-light-modulators, this all-optical deep learning framework can perform, at the speed of light, various complex functions that computer-based neural networks can implement, and will find applications in all-optical image analysis, feature detection and object classification, also enabling new camera designs and optical components that can learn to perform unique tasks using D2NNs.

/pdf/all-optical-machine-learning-using-diffractive-deep-neural-2fuy5hs9sa.pdf

All-Optical Machine Learning Using Diffractive Deep Neural Networks

There is an ever-growing demand for artificial intelligence. Optical processors, which compute with photons instead of electrons, can fundamentally accelerate the development of artificial intelligence by offering substantially improved computing performance. There has been long-term interest in optically constructing the most widely used artificial-intelligence architecture, that is, artificial neural networks, to achieve brain-inspired information processing at the speed of light. However, owing to restrictions in design flexibility and the accumulation of system errors, existing processor architectures are not reconfigurable and have limited model complexity and experimental performance. Here, we propose the reconfigurable diffractive processing unit, an optoelectronic fused computing architecture based on the diffraction of light, which can support different neural networks and achieve a high model complexity with millions of neurons. Along with the developed adaptive training approach to circumvent system errors, we achieved excellent experimental accuracies for high-speed image and video recognition over benchmark datasets and a computing performance superior to that of cutting-edge electronic computing platforms. Linear diffractive structures are by themselves passive systems but researchers here exploit the non-linearity of a photodetector to realize a reconfigurable diffractive ‘processing’ unit. High-speed image and video recognition is demonstrated.

Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit

This paper proposes a novel compressive hyperspectral (HS) imaging approach that allows for high-resolution HS images to be captured in a single image. The proposed architecture comprises three key components: spatial-spectral encoded optical camera design, over-complete HS dictionary learning and sparse-constraint computational reconstruction. Our spatial-spectral encoded sampling scheme provides a higher degree of randomness in the measured projections than previous compressive HS imaging approaches; and a robust nonlinear sparse reconstruction method is employed to recover the HS images from the coded projection with higher performance. To exploit the sparsity constraint on the nature HS images for computational reconstruction, an over-complete HS dictionary is learned to represent the HS images in a sparser way than previous representations. We validate the proposed approach on both synthetic and real captured data, and show successful recovery of HS images for both indoor and outdoor scenes. In addition, we demonstrate other applications for the over-complete HS dictionary and sparse coding techniques, including 3D HS images compression and denoising.

Spatial-spectral encoded compressive hyperspectral imaging

Multispectral cameras collect image data with a greater number of spectral channels than traditional trichromatic sensors, thus providing spectral information at a higher level of detail. Such data are useful in various fields, such as remote sensing, materials science, biophotonics, and environmental monitoring. The massive scale of multispectral data-at high resolutions in the spectral, spatial, and temporal dimensions-has long presented a major challenge in spectrometer design. With recent developments in sampling theory, this problem has become more manageable through use of undersampling and constrained reconstruction techniques. This article presents an overview of these state-of-the-art multispectral acquisition systems, with a particular focus on snapshot multispectral capture, from a signal processing perspective. We propose that undersampling-based multispectral cameras can be understood and compared by examining the efficiency of their sampling schemes, which we formulate as the spectral sensing coherence information between their sensing matrices and spectrum-specific bases learned from a large-scale multispectral image database. We analyze existing snapshot multispectral cameras in this manner, and additionally discuss their optical performance in terms of light throughput and system complexity.

Xing Lin

Papers

All-optical machine learning using diffractive deep neural networks

All-Optical Machine Learning Using Diffractive Deep Neural Networks

Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit

Spatial-spectral encoded compressive hyperspectral imaging

Computational Snapshot Multispectral Cameras: Toward dynamic capture of the spectral world