Optical neural networks (ONNs) have become competitive candidates for the next generation of high-performance neural network accelerators because of their low power consumption and high-speed nature. Beyond fully-connected neural networks demonstrated in pioneer works, optical computing hardwares can also conduct convolutional neural networks (CNNs) by hardware reusing. Following this concept, we propose an optical convolution unit (OCU) architecture. By reusing the OCU architecture with different inputs and weights, convolutions with arbitrary input sizes can be done. A proof-of-concept experiment is carried out by cascaded acousto-optical modulator arrays. When the neural network parameters are ex-situ trained, the OCU conducts convolutions with SDR up to 28.22 dBc and performs well on inferences of typical CNN tasks. Furthermore, we conduct in situ training and get higher SDR at 36.27 dBc, verifying the OCU could be further refined by in situ training. Besides the effectiveness and high accuracy, the simplified OCU architecture served as a building block could be easily duplicated and integrated to future chip-scale optical CNNs.

High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays

The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era. Photonics neuromorphic computing has attracted lots of attention due to the fascinating advantages such as high speed, wide bandwidth, and massive parallelism. Here, we offer a review on the optical neural computing in our research groups at the device and system levels. The photonics neuron and photonics synapse plasticity are presented. In addition, we introduce several optical neural computing architectures and algorithms including photonic spiking neural network, photonic convolutional neural network, photonic matrix computation, photonic reservoir computing, and photonic reinforcement learning. Finally, we summarize the major challenges faced by photonic neuromorphic computing, and propose promising solutions and perspectives.

A review: Photonics devices, architectures, and algorithms for optical neural computing

https://onlinelibrary.wiley.com/doi/pdf/10.1002/adpr.202000212

Integrated Neuromorphic Photonics: Synapses, Neurons, and Neural Networks

In the last decades, photonic technologies have been increasingly proposed for microwave and millimeter-wave applications, opening the way to the development of a new field of research known as Microwave Photonics. The hybridization of the microwave world with photonics has demonstrated a beneficial impact on both communications and remote sensing. 5G radio access networks, as well as the next-coming generation of distributed, multistatic radars, are expected to massively leverage on photonic techniques for the generation, distribution, processing, and acquisition of microwave signals. This is due to the advantages brought about by photonics in terms of system transparency to the employed frequency or waveform, low losses, electromagnetic interference immunity, high signal stability. In particular, the inherent coherence guaranteed by photonics will enable next-future multiple input-multiple output radars with enhanced performance. In this paper, we expose in detail some of the most widely employed functions on RF signals obtained with photonic techniques, highlighting the achievable performance that help overcoming some limitations of classical electronic technologies. Finally, two recently implemented microwave photonics systems are described, i.e., a multiple input-multiple output radar and a RF spectrum scanner, showing with practical examples all the potential of microwave photonics systems.

Microwave Photonics for Remote Sensing: From Basic Concepts to High-Level Functionalities

Recent progress on optical neural networks (ONNs) heralds a new future for efficient deep learning accelerators, and novel, to the best of our knowledge, architectures of optical convolutional neural networks (CNNs) provide potential solutions to the widely adopted convolutional models. So far in optical CNNs, the data patching (a necessary process in the convolutional layer) is mostly executed with electronics, resulting in a demand for large input modulator arrays. Here we experimentally demonstrate an optical patching scheme to release the burden of electronic data processing and to cut down the scale of the input modulator array for optical CNNs. Optical delay lines replace electronics to execute data processing, which can reduce the scale of the input modulator array. The adoption of wavelength-division multiplexing enables a single group of optical delay lines to simultaneously process multiple input data, reducing the system complexity. The optical patching scheme provides a new solution to the problem of data input, which is challenging and concerned with the field of ONNs.

Optical patching scheme for optical convolutional neural networks based on wavelength-division multiplexing and optical delay lines

Analog-to-digital converters (ADCs) must be high speed, broadband, and accurate for the development of modern information systems, such as radar, imaging, and communications systems; photonic technologies are regarded as promising technologies for realizing these advanced requirements. Here, we present a deep-learning-powered photonic ADC architecture that simultaneously exploits the advantages of electronics and photonics and overcomes the bottlenecks of the two technologies, thereby overcoming the ADC tradeoff among speed, bandwidth, and accuracy. Via supervised training, the adopted deep neural networks learn the patterns of photonic system defects and recover the distorted data, thereby maintaining the high quality of the electronic quantized data succinctly and adaptively. The numerical and experimental results demonstrate that the proposed architecture outperforms state-of-the-art ADCs with developable high throughput; hence, deep learning performs well in photonic ADC systems. We anticipate that the proposed architecture will inspire future high-performance photonic ADC design and provide opportunities for substantial performance enhancement for the next-generation information systems.

/pdf/deep-learning-powered-photonic-analog-to-digital-conversion-21041015rb.pdf

Deep-learning-powered photonic analog-to-digital conversion.

The Brillouin instantaneous frequency measurement (B-IFM) is used to measure instantaneous frequencies of an arbitrary signal with high frequency and broad bandwidth. However, the instantaneous frequencies measured using the B-IFM system always suffer from errors, due to system defects. To address this, we adopt a convolutional neural network (CNN) that establishes a function mapping between the measured and nominal instantaneous frequencies to obtain a more accurate instantaneous frequency, thus improving the frequency resolution, system sensitivity, and dynamic range of the B-IFM. Using the proposed CNN-optimized B-IFM system, the average maximum and root mean square errors between the optimized and nominal instantaneous frequencies are less than 26.3 and 15.5 MHz, which is reduced from up to 105.8 and 57.0 MHz. The system sensitivity is increased from 12.1 to 7.8 dBm for the 100 MHz frequency error, and the dynamic range is larger.

Optimization of the Brillouin instantaneous frequency measurement using convolutional neural networks.

The emerging intelligence technologies represented by deep learning have broadened their applications to various fields. Beyond the conventional electronics-based processing systems, the convergence of photonics and artificial intelligence (AI) technology enhances the performance and learning ability of AI. In this review, we propose the concept of an intelligent photonic system (IPS), illustrating it as a developing architecture with three different versions. For each version of IPS, we review several representative studies. Moreover we discuss the challenges towards an IPS and provide some prospects for the future development.

Towards an intelligent photonic system

We propose a convolutional recurrent autoencoder (CRAE) to compensate for time mismatches in a photonic analog-to-digital converter (PADC). In contrast of other neural networks, the proposed CRAE is generalized to untrained mismatches and untrained category of signals while remaining robust to system states. We train the CRAE using mismatched linear frequency modulated (LFM) signals with mismatches of 35 ps and 57 ps under one system state. It can effectively compensate for mismatches of both LFM and Costas frequency modulated signals with mismatches ranging from 35 ps to 137 ps under another system state. When the spur-free dynamic range (SFDR) of the unpowered PADC decreases from 10.2 dBc to -3.0 dBc, the SFDR of the CRAE-powered PADC is over 31.6 dBc.

Photonic analog-to-digital converter powered by a generalized and robust convolutional recurrent autoencoder

As the bridge between the analog world and digital computers, analog-to-digital converters are generally used in modern information systems such as radar, surveillance, and communications. For the configuration of analog-to-digital converters in future high-frequency broadband systems, we introduce a revolutionary architecture that adopts deep learning technology to overcome tradeoffs between bandwidth, sampling rate, and accuracy. A photonic front-end provides broadband capability for direct sampling and speed multiplication. Trained deep neural networks learn the patterns of system defects, maintaining high accuracy of quantized data in a succinct and adaptive manner. Based on numerical and experimental demonstrations, we show that the proposed architecture outperforms state-of-the-art analog-to-digital converters, confirming the potential of our approach in future analog-to-digital converter design and performance enhancement of future information systems.

Xiuting Zou

Papers

Deep-learning-powered photonic analog-to-digital conversion.

Optimization of the Brillouin instantaneous frequency measurement using convolutional neural networks.

Towards an intelligent photonic system

Photonic analog-to-digital converter powered by a generalized and robust convolutional recurrent autoencoder

Analog-to-digital conversion revolutionized by deep learning