Li Jing

Bio: Li Jing is an academic researcher from Massachusetts Institute of Technology. The author has contributed to research in topics: Artificial neural network & Recurrent neural network. The author has an hindex of 17, co-authored 56 publications receiving 1315 citations. Previous affiliations of Li Jing include Peking University & Facebook.

Nanophotonic particle simulation and inverse design using artificial neural networks

Abstract: We propose a method to use artificial neural networks to approximate light scattering by multilayer nanoparticles. We find that the network needs to be trained on only a small sampling of the data to approximate the simulation to high precision. Once the neural network is trained, it can simulate such optical processes orders of magnitude faster than conventional simulations. Furthermore, the trained neural network can be used to solve nanophotonic inverse design problems by using back propagation, where the gradient is analytical, not numerical.

Deep-learning-enabled self-adaptive microwave cloak without human intervention

Abstract: Becoming invisible at will has fascinated humanity for centuries and in the past decade it has attracted a great deal of attention owing to the advent of metamaterials. However, state-of-the-art invisibility cloaks typically work in a deterministic system or in conjunction with outside help to achieve active cloaking. Here, we propose the concept of an intelligent (that is, self-adaptive) cloak driven by deep learning and present a metasurface cloak as an example implementation. In the experiment, the metasurface cloak exhibits a millisecond response time to an ever-changing incident wave and the surrounding environment, without any human intervention. Our work brings the available cloaking strategies closer to a wide range of real-time, in situ applications, such as moving stealth vehicles. The approach opens the way to facilitating other intelligent metadevices in the microwave regime and across the wider electromagnetic spectrum and, more generally, enables automatic solutions of electromagnetic inverse design problems. A deep-learning-enabled metasurface cloak actively self-adapts to take into account changing microwave illumination and varying physical surroundings.

Nanophotonic particle simulation and inverse design using artificial neural networks

Abstract: We propose a method to use artificial neural networks to approximate light scattering by multilayer nanoparticles. We find the network needs to be trained on only a small sampling of the data in order to approximate the simulation to high precision. Once the neural network is trained, it can simulate such optical processes orders of magnitude faster than conventional simulations. Furthermore, the trained neural network can be used solve nanophotonic inverse design problems by using back-propogation - where the gradient is analytical, not numerical.

Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs

Abstract: Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, we present a new architecture for implementing an Efficient Unitary Neural Network (EUNNs); its main advantages can be summarized as follows. Firstly, the representation capacity of the unitary space in an EUNN is fully tunable, ranging from a subspace of SU(N) to the entire unitary space. Secondly, the computational complexity for training an EUNN is merely O(1) per parameter. Finally, we test the performance of EUNNs on the standard copying task, the pixel-permuted MNIST digit recognition benchmark as well as the Speech Prediction Test (TIMIT). We find that our architecture significantly outperforms both other state-of-the-art unitary RNNs and the LSTM architecture, in terms of the final performance and/or the wall-clock training speed. EUNNs are thus promising alternatives to RNNs and LSTMs for a wide variety of applications.

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

Abstract: For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling. The models are evaluated across a broad range of standard tasks that are commonly used to benchmark recurrent networks. Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory. We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling tasks. To assist related work, we have made code available at this http URL .

A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

Abstract: Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are...

Regularizing and Optimizing LSTM Language Models

Abstract: Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine translation, language modeling, and question answering. In this paper, we consider the specific problem of word-level language modeling and investigate strategies for regularizing and optimizing LSTM-based models. We propose the weight-dropped LSTM which uses DropConnect on hidden-to-hidden weights as a form of recurrent regularization. Further, we introduce NT-ASGD, a variant of the averaged stochastic gradient method, wherein the averaging trigger is determined using a non-monotonic condition as opposed to being tuned by the user. Using these and other regularization strategies, we achieve state-of-the-art word level perplexities on two data sets: 57.3 on Penn Treebank and 65.8 on WikiText-2. In exploring the effectiveness of a neural cache in conjunction with our proposed model, we achieve an even lower state-of-the-art perplexity of 52.8 on Penn Treebank and 52.0 on WikiText-2.

Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures

Abstract: Data inconsistency leads to a slow training process when deep neural networks are used for the inverse design of photonic devices, an issue that arises from the fundamental property of nonuniqueness in all inverse scattering problems. Here we show that by combining forward modeling and inverse design in a tandem architecture, one can overcome this fundamental issue, allowing deep neural networks to be effectively trained by data sets that contain nonunique electromagnetic scattering instances. This paves the way for using deep neural networks to design complex photonic structures that require large training data sets.