Domain-specific hardware is becoming a promising topic in the backdrop of improvement slow down for general-purpose processors due to the foreseeable end of Moore’s Law. Machine learning, especially deep neural networks (DNNs), has become the most dazzling domain witnessing successful applications in a wide spectrum of artificial intelligence (AI) tasks. The incomparable accuracy of DNNs is achieved by paying the cost of hungry memory consumption and high computational complexity, which greatly impedes their deployment in embedded systems. Therefore, the DNN compression concept was naturally proposed and widely used for memory saving and compute acceleration. In the past few years, a tremendous number of compression techniques have sprung up to pursue a satisfactory tradeoff between processing efficiency and application accuracy. Recently, this wave has spread to the design of neural network accelerators for gaining extremely high performance. However, the amount of related works is incredibly huge and the reported approaches are quite divergent. This research chaos motivates us to provide a comprehensive survey on the recent advances toward the goal of efficient compression and execution of DNNs without significantly compromising accuracy, involving both the high-level algorithms and their applications in hardware design. In this article, we review the mainstream compression approaches such as compact model, tensor decomposition, data quantization, and network sparsification. We explain their compression principles, evaluation metrics, sensitivity analysis, and joint-way use. Then, we answer the question of how to leverage these methods in the design of neural network accelerators and present the state-of-the-art hardware architectures. In the end, we discuss several existing issues such as fair comparison, testing workloads, automatic compression, influence on security, and framework/hardware-level support, and give promising topics in this field and the possible challenges as well. This article attempts to enable readers to quickly build up a big picture of neural network compression and acceleration, clearly evaluate various methods, and confidently get started in the right way.

Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey

We present the Vortex Image Processing (VIP) library, a python package dedicated to astronomical high-contrast imaging Our package relies on the extensive python stack of scientific libraries and aims to provide a flexible framework for high-contrast data and image processing In this paper, we describe the capabilities of VIP related to processing image sequences acquired using the angular differential imaging (ADI) observing technique VIP implements functionalities for building high-contrast data processing pipelines, encompassing pre- and post-processing algorithms, potential source position and flux estimation, and sensitivity curve generation Among the reference point-spread function subtraction techniques for ADI post-processing, VIP includes several flavors of principal component analysis (PCA) based algorithms, such as annular PCA and incremental PCA algorithms capable of processing big datacubes (of several gigabytes) on a computer with limited memory Also, we present a novel ADI algorithm based on non-negative matrix factorization, which comes from the same family of low-rank matrix approximations as PCA and provides fairly similar results We showcase the ADI capabilities of the VIP library using a deep sequence on HR 8799 taken with the LBTI/LMIRCam and its recently commissioned L-band vortex coronagraph Using VIP, we investigated the presence of additional companions around HR 8799 and did not find any significant additional point source beyond the four known planets VIP is available at http://githubcom/vortex-exoplanet/VIP and is accompanied with Jupyter notebook tutorials illustrating the main functionalities of the library

/pdf/vip-vortex-image-processing-package-for-high-contrast-direct-4cquetxkft.pdf

VIP: Vortex Image Processing Package for High-contrast Direct Imaging

Recently, deep learning models have shown promising performances in many research areas, including traffic states prediction, due to their ability to model complex nonlinear relationships. However, deep learning models also have drawbacks that make them less preferable for certain short-term traffic prediction applications. For example, they require a large amount of data for model training, which is also computationally expensive. Moreover, deep learning models lack interpretability of the results. This paper develops a short-term traffic states forecasting algorithm based on partial least square (PLS) to help enhance real-time decision-making and build better insights into traffic data. The proposed model is capable of predicting short-term traffic states accurately and efficiently by capturing dominant spatiotemporal features and day-to-day variations from collinear and correlated traffic data. Three case studies are developed to demonstrate the proposed model in short-term traffic prediction applications.

Short-term traffic state prediction from latent structures: Accuracy vs. efficiency

In this work we introduce a new multiscale artificial neural network based on the structure of $\mathcal{H}$-matrices. This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale. Numerical results indicate that the network is able to efficiently approximate discrete nonlinear maps obtained from discretized nonlinear partial differential equations, such as those arising from nonlinear Schrodinger equations and the Kohn-Sham density functional theory.

A multiscale neural network based on hierarchical matrices

Objective Epilepsy is one of the most common brain disorders. For epilepsy diagnosis or treatment, the neurologist needs to observe epileptic spikes from electroencephalography (EEG) data. Since multi-channel EEG records can be naturally represented by multi-way tensors, it is of interest to see whether tensor decomposition is able to analyze EEG epileptic spikes. Approach In this paper, we first proposed the problem of simultaneous multilinear low-rank approximation of tensors (SMLRAT) and proved that SMLRAT can obtain local optimum solutions by using two well-known tensor decomposition algorithms (HOSVD and Tucker-ALS). Second, we presented a new system for automatic epileptic spike detection based on SMLRAT. Main results We propose to formulate the problem of feature extraction from a set of EEG segments, represented by tensors, as the SMLRAT problem. Efficient EEG features were obtained, based on estimating the 'eigenspikes' derived from nonnegative GSMLRAT. We compared the proposed tensor analysis method with other common tensor methods in analyzing EEG signal and compared the proposed feature extraction method with the state-of-the-art methods. Experimental results indicated that our proposed method is able to detect epileptic spikes with high accuracy. Significance Our method, for the first time, makes a step forward for automatic detection EEG epileptic spikes based on tensor decomposition. The method can provide a practical solution to distinguish epileptic spikes from artifacts in real-life EEG datasets.

https://iopscience.iop.org/article/10.1088/1741-2552/ab5247/pdf

Multi-channel EEG epileptic spike detection by a new method of tensor decomposition.

Low rank approximation of matrices has been well studied in literature. Singular value decomposition, QR decomposition with column pivoting, rank revealing QR factorization, Interpolative decomposi...

Literature survey on low rank approximation of matrices

Low rank approximation of matrices has been well studied in literature. Singular value decomposition, QR decomposition with column pivoting, rank revealing QR factorization (RRQR), Interpolative decomposition etc are classical deterministic algorithms for low rank approximation. But these techniques are very expensive $(O(n^{3})$ operations are required for $n\times n$ matrices). There are several randomized algorithms available in the literature which are not so expensive as the classical techniques (but the complexity is not linear in n). So, it is very expensive to construct the low rank approximation of a matrix if the dimension of the matrix is very large. There are alternative techniques like Cross/Skeleton approximation which gives the low-rank approximation with linear complexity in n . In this article we review low rank approximation techniques briefly and give extensive references of many techniques.

In this paper we show that we can use a modified version of the h-p spectral element method proposed in [6,7,13,14] to solve elliptic problems with general boundary conditions to exponential accuracy on polygonal domains using nonconforming spectral element functions. A geometrical mesh is used in a neighbourhood of the corners. With this mesh we seek a solution which minimizes the sum of a weighted squared norm of the residuals in the partial differential equation and the squared norm of the residuals in the boundary conditions in fractional Sobolev spaces and enforce continuity by adding a term which measures the jump in the function and its derivatives at inter-element boundaries, in fractional Sobolev norms, to the functional being minimized. In the neighbourhood of the corners, modified polar coordinates are used and a global coordinate system elsewhere. A stability estimate is derived for the functional which is minimized based on the regularity estimate in [2]. We examine how to parallelize the method and show that the set of common boundary values consists of the values of the function at the corners of the polygonal domain. The method is faster than that proposed in [6,7,14] and the h-p finite element method and stronger error estimates are obtained.

Nonconforming h-p spectral element methods for elliptic problems

In this paper we show that we can use a modified version of the h-p spectral element method proposed in \cite{duttora1,duttom,duttora2,tomarth} to solve elliptic problems with general boundary conditions to exponential accuracy on polygonal domains using nonconforming spectral element functions. A geometrical mesh is used in a neighbourhood of the corners. With this mesh we seek a solution which minimizes the sum of a weighted squared norm of the residuals in the partial differential equation and the squared norm of the residuals in the boundary conditions in fractional Sobolev spaces and enforce continuity by adding a term which measures the jump in the function and its derivatives at inter-element boundaries, in fractional Sobolev norms, to the functional being minimized. In the neighbourhood of the corners, modified polar coordinates are used and a global coordinate system elsewhere. A stability estimate is derived for the functional which is minimized based on the regularity estimate in \cite{babguo1}. We examine how to parallelize the method and show that the set of common boundary values consists of the values of the function at the corners of the polygonal domain. The method is faster than that proposed in \cite{duttora1,duttom,tomarth} and the h-p finite element method and stronger error estimates are obtained.

In this paper a least-squares based method is proposed for elliptic interface problems in two dimensions, where the interface is smooth. The underlying method is spectral element method. The least-squares formulation is based on the minimization of a functional as defined in (4.1). The jump in the solution and its normal derivative across the interface are enforced (in an appropriate Sobolev norm) in the functional. The solution is obtained by solving the normal equations using preconditioned conjugate gradient method. Essentially the method is nonconforming, so a block diagonal matrix is constructed as a preconditioner based on the stability estimate where each diagonal block is decoupled. A conforming solution is obtained by making a set of corrections to the nonconforming solution as in Schwab (p and h---p Finite Element Methods, Clarendon Press, Oxford, 1998) and an error estimate in H 1-norm is given which shows the exponential convergence of the proposed method.

/pdf/nonconforming-least-squares-method-for-elliptic-partial-4l6bftmbsb.pdf

N. Kishore Kumar

Papers

Literature survey on low rank approximation of matrices

Literature survey on low rank approximation of matrices

Nonconforming h-p spectral element methods for elliptic problems

Nonconforming h-p spectral element methods for elliptic problems

Nonconforming Least-Squares Method for Elliptic Partial Differential Equations with Smooth Interfaces