Showing papers on "Matrix (mathematics) published in 2018"

Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec

Abstract: Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE can be viewed as the joint factorization of multiple networks» Laplacians; (4) node2vec is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.

OCNet: Object Context Network for Scene Parsing

Abstract: In this paper, we address the semantic segmentation task with a new context aggregation scheme named \emph{object context}, which focuses on enhancing the role of object information. Motivated by the fact that the category of each pixel is inherited from the object it belongs to, we define the object context for each pixel as the set of pixels that belong to the same category as the given pixel in the image. We use a binary relation matrix to represent the relationship between all pixels, where the value one indicates the two selected pixels belong to the same category and zero otherwise. We propose to use a dense relation matrix to serve as a surrogate for the binary relation matrix. The dense relation matrix is capable to emphasize the contribution of object information as the relation scores tend to be larger on the object pixels than the other pixels. Considering that the dense relation matrix estimation requires quadratic computation overhead and memory consumption w.r.t. the input size, we propose an efficient interlaced sparse self-attention scheme to model the dense relations between any two of all pixels via the combination of two sparse relation matrices. To capture richer context information, we further combine our interlaced sparse self-attention scheme with the conventional multi-scale context schemes including pyramid pooling~\citep{zhao2017pyramid} and atrous spatial pyramid pooling~\citep{chen2018deeplab}. We empirically show the advantages of our approach with competitive performances on five challenging benchmarks including: Cityscapes, ADE20K, LIP, PASCAL-Context and COCO-Stuff

Predicting station-level hourly demand in a large-scale bike-sharing network: A graph convolutional neural network approach

Abstract: This study proposes a novel Graph Convolutional Neural Network with Data-driven Graph Filter (GCNN-DDGF) model that can learn hidden heterogeneous pairwise correlations between stations to predict station-level hourly demand in a large-scale bike-sharing network. Two architectures of the GCNN-DDGF model are explored; GCNNreg-DDGF is a regular GCNN-DDGF model which contains the convolution and feedforward blocks, and GCNNrec-DDGF additionally contains a recurrent block from the Long Short-term Memory neural network architecture to capture temporal dependencies in the bike-sharing demand series. Furthermore, four types of GCNN models are proposed whose adjacency matrices are based on various bike-sharing system data, including Spatial Distance matrix (SD), Demand matrix (DE), Average Trip Duration matrix (ATD), and Demand Correlation matrix (DC). These six types of GCNN models and seven other benchmark models are built and compared on a Citi Bike dataset from New York City which includes 272 stations and over 28 million transactions from 2013 to 2016. Results show that the GCNNrec-DDGF performs the best in terms of the Root Mean Square Error, the Mean Absolute Error and the coefficient of determination (R2), followed by the GCNNreg-DDGF. They outperform the other models. Through a more detailed graph network analysis based on the learned DDGF, insights are obtained on the “black box” of the GCNN-DDGF model. It is found to capture some information similar to details embedded in the SD, DE and DC matrices. More importantly, it also uncovers hidden heterogeneous pairwise correlations between stations that are not revealed by any of those matrices.

Optimal Whitening and Decorrelation

Abstract: Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example based on principal component analysis (PCA), Cholesky matrix decomposition and zero-phase component analysis (ZCA), among others. Here we provide an overview of the underlying theory and discuss five natural whitening procedures. Subsequently, we demonstrate that investigating the cross-covariance and the cross-correlation matrix between sphered and original variables allows to break the rotational invariance and to identify optimal whitening transformations. As a result we recommend two particular approaches: ZCA-cor whitening to produce sphered variables that are maximally similar to the original variables, and PCA-cor whitening to obtain sphered variables that maximally compress the original variables.

Cell contraction induces long-ranged stress stiffening in the extracellular matrix

Abstract: Animal cells in tissues are supported by biopolymer matrices, which typically exhibit highly nonlinear mechanical properties. While the linear elasticity of the matrix can significantly impact cell mechanics and functionality, it remains largely unknown how cells, in turn, affect the nonlinear mechanics of their surrounding matrix. Here, we show that living contractile cells are able to generate a massive stiffness gradient in three distinct 3D extracellular matrix model systems: collagen, fibrin, and Matrigel. We decipher this remarkable behavior by introducing nonlinear stress inference microscopy (NSIM), a technique to infer stress fields in a 3D matrix from nonlinear microrheology measurements with optical tweezers. Using NSIM and simulations, we reveal large long-ranged cell-generated stresses capable of buckling filaments in the matrix. These stresses give rise to the large spatial extent of the observed cell-induced matrix stiffness gradient, which can provide a mechanism for mechanical communication between cells.

Subspace Clustering by Block Diagonal Representation

Abstract: This paper studies the subspace clustering problem. Given some data points approximately drawn from a union of subspaces, the goal is to group these data points into their underlying subspaces. Many subspace clustering methods have been proposed and among which sparse subspace clustering and low-rank representation are two representative ones. Despite the different motivations, we observe that many existing methods own the common block diagonal property, which possibly leads to correct clustering, yet with their proofs given case by case. In this work, we consider a general formulation and provide a unified theoretical guarantee of the block diagonal property. The block diagonal property of many existing methods falls into our special case. Second, we observe that many existing methods approximate the block diagonal representation matrix by using different structure priors, e.g., sparsity and low-rankness, which are indirect. We propose the first block diagonal matrix induced regularizer for directly pursuing the block diagonal matrix. With this regularizer, we solve the subspace clustering problem by Block Diagonal Representation (BDR), which uses the block diagonal structure prior. The BDR model is nonconvex and we propose an alternating minimization solver and prove its convergence. Experiments on real datasets demonstrate the effectiveness of BDR.

Secure Outsourced Matrix Computation and Application to Neural Networks

Abstract: Homomorphic Encryption (HE) is a powerful cryptographic primitive to address privacy and security issues in outsourcing computation on sensitive data to an untrusted computation environment Comparing to secure Multi-Party Computation (MPC), HE has advantages in supporting non-interactive operations and saving on communication costs However, it has not come up with an optimal solution for modern learning frameworks, partially due to a lack of efficient matrix computation mechanisms In this work, we present a practical solution to encrypt a matrix homomorphically and perform arithmetic operations on encrypted matrices Our solution includes a novel matrix encoding method and an efficient evaluation strategy for basic matrix operations such as addition, multiplication, and transposition We also explain how to encrypt more than one matrix in a single ciphertext, yielding better amortized performance Our solution is generic in the sense that it can be applied to most of the existing HE schemes It also achieves reasonable performance for practical use; for example, our implementation takes 921 seconds to multiply two encrypted square matrices of order 64 and 256 seconds to transpose a square matrix of order 64 Our secure matrix computation mechanism has a wide applicability to our new framework EDM, which stands for encrypted data and encrypted model To the best of our knowledge, this is the first work that supports secure evaluation of the prediction phase based on both encrypted data and encrypted model, whereas previous work only supported applying a plain model to encrypted data As a benchmark, we report an experimental result to classify handwritten images using convolutional neural networks (CNN) Our implementation on the MNIST dataset takes 2859 seconds to compute ten likelihoods of 64 input images simultaneously, yielding an amortized rate of 045 seconds per image

A Novel Robust Fuzzy Integral Sliding Mode Control for Nonlinear Semi-Markovian Jump T–S Fuzzy Systems

Abstract: This paper addresses the issue of robust fuzzy sliding mode control for continuous-time nonlinear Takagi–Sugeno fuzzy systems with semi-Markovian switching. The focus is on designing a novel fuzzy integral sliding surface without assuming that the input matrices are the same with full column rank and then developing a fuzzy sliding-mode controller for stochastic stability purpose. Based on Lyapunov theory, a set of newly developed linear matrix inequality conditions are established for stochastic stability of the sliding-mode dynamics with generally uncertain transition rates, and then extended to where the input matrix is plant-rule-independent, as discussed in most existing literatures. Furthermore, finite-time reachability of the sliding surface is also guaranteed by the proposed fuzzy sliding-mode control laws. A practical example is provided to demonstrate the effectiveness of the established method numerically.

Size-Independent Sample Complexity of Neural Networks

Abstract: We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth, and under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.

Fully differential NNLO computations with MATRIX

Abstract: We present the computational framework Matrix ( http://matrix.hepforge.org/ ) which allows us to evaluate fully differential cross sections for a wide class of processes at hadron colliders in next-to-next-to-leading order (NNLO) QCD. The processes we consider are $$2\rightarrow 1$$ and $$2\rightarrow 2$$ hadronic reactions involving Higgs and vector bosons in the final state. All possible leptonic decay channels of the vector bosons are included for the first time in the calculations, by consistently accounting for all resonant and non-resonant diagrams, off-shell effects and spin correlations. We briefly introduce the theoretical framework Matrix is based on, discuss its relevant features and provide a detailed description of how to use Matrix to obtain NNLO accurate results for the various processes. We report reference predictions for inclusive and fiducial cross sections of all the physics processes considered here and discuss their corresponding uncertainties. Matrix features an automatic extrapolation procedure that allows us, for the first time, to control the systematic uncertainties inherent to the applied NNLO subtraction procedure down to the few permille level (or better).

RNN Models for Dynamic Matrix Inversion: A Control-Theoretical Perspective

Abstract: In this paper, the existing recurrent neural network (RNN) models for solving zero-finding (e.g., matrix inversion) with time-varying parameters are revisited from the perspective of control and unified into a control-theoretical framework. Then, limitations on the activated functions of existing RNN models are pointed out and remedied with the aid of control-theoretical techniques. In addition, gradient-based RNNs, as the classical method for zero-finding, have been remolded to solve dynamic problems in manners free of errors and matrix inversions. Finally, computer simulations are conducted and analyzed to illustrate the efficacy and superiority of the modified RNN models designed from the perspective of control. The main contribution of this paper lies in the removal of the convex restriction and the elimination of the matrix inversion in existing RNN models for the dynamic matrix inversion. This work provides a systematic approach on exploiting control techniques to design RNN models for robustly and accurately solving algebraic equations.

High dimensional change point estimation via sparse projection

Abstract: Summary Change points are a very common feature of ‘big data’ that arrive in the form of a data stream. We study high dimensional time series in which, at certain time points, the mean structure changes in a sparse subset of the co-ordinates. The challenge is to borrow strength across the co-ordinates to detect smaller changes than could be observed in any individual component series. We propose a two-stage procedure called inspect for estimation of the change points: first, we argue that a good projection direction can be obtained as the leading left singular vector of the matrix that solves a convex optimization problem derived from the cumulative sum transformation of the time series. We then apply an existing univariate change point estimation algorithm to the projected series. Our theory provides strong guarantees on both the number of estimated change points and the rates of convergence of their locations, and our numerical studies validate its highly competitive empirical performance for a wide range of data-generating mechanisms. Software implementing the methodology is available in the R package InspectChangepoint.

Linear Convergence in Optimization Over Directed Graphs With Row-Stochastic Matrices

Abstract: This paper considers a distributed optimization problem over a multiagent network, in which the objective function is a sum of individual cost functions at the agents. We focus on the case when communication between the agents is described by a directed graph. Existing distributed optimization algorithms for directed graphs require at least the knowledge of the neighbors’ out-degree at each agent (due to the requirement of column-stochastic matrices). In contrast, our algorithm requires no such knowledge. Moreover, the proposed algorithm achieves the best known rate of convergence for this class of problems, $O(\mu ^k)$ for $0 , where $k$ is the number of iterations, given that the objective functions are strongly convex and have Lipschitz-continuous gradients. Numerical experiments are also provided to illustrate the theoretical findings.

A quantum-inspired classical algorithm for recommendation systems

Abstract: We give a classical analogue to Kerenidis and Prakash's quantum recommendation system, previously believed to be one of the strongest candidates for provably exponential speedups in quantum machine learning Our main result is an algorithm that, given an $m \times n$ matrix in a data structure supporting certain $\ell^2$-norm sampling operations, outputs an $\ell^2$-norm sample from a rank-$k$ approximation of that matrix in time $O(\text{poly}(k)\log(mn))$, only polynomially slower than the quantum algorithm As a consequence, Kerenidis and Prakash's algorithm does not in fact give an exponential speedup over classical algorithms Further, under strong input assumptions, the classical recommendation system resulting from our algorithm produces recommendations exponentially faster than previous classical systems, which run in time linear in $m$ and $n$ The main insight of this work is the use of simple routines to manipulate $\ell^2$-norm sampling distributions, which play the role of quantum superpositions in the classical setting This correspondence indicates a potentially fruitful framework for formally comparing quantum machine learning algorithms to classical machine learning algorithms

A review of recent research on materials used in polymer–matrix composites for body armor application:

Abstract: The development of personal protection systems with improved ballistic performance and reduced weight has received a great interest in the last decade with the unfortunate ever-increasing threats a...

Robust Covariance and Scatter Matrix Estimation under Huber's Contamination Model

Abstract: Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

Viscoplastic Matrix Materials for Embedded 3D Printing

Abstract: Embedded three-dimensional (EMB3D) printing is an emerging technique that enables free-form fabrication of complex architectures. In this approach, a nozzle is translated omnidirectionally within a soft matrix that surrounds and supports the patterned material. To optimize print fidelity, we have investigated the effects of matrix viscoplasticity on the EMB3D printing process. Specifically, we determine how matrix composition, print path and speed, and nozzle diameter affect the yielded region within the matrix. By characterizing the velocity and strain fields and analyzing the dimensions of the yielded regions, we determine that scaling relationships based on the Oldroyd number, Od, exist between these dimensions and the rheological properties of the matrix materials and printing parameters. Finally, we use EMB3D printing to create complex architectures within an elastomeric silicone matrix. Our methods and findings will both facilitate future characterization of viscoplastic matrices and motivate the development of new materials for EMB3D printing.

Improved rectangular matrix multiplication using powers of the coppersmith-winograd tensor

Abstract: In the past few years, successive improvements of the asymptotic complexity of square matrix multiplication have been obtained by developing novel methods to analyze the powers of the Coppersmith-Winograd tensor, a basic construction introduced thirty years ago. In this paper we show how to generalize this approach to make progress on the complexity of rectangular matrix multiplication as well, by developing a framework to analyze powers of tensors in an asymmetric way. By applying this methodology to the fourth power of the Coppersmith-Winograd tensor, we succeed in improving the complexity of rectangular matrix multiplication. Let α denote the maximum value such that the product of an n × nα matrix by an nα × n matrix can be computed with O(n2+ϵ) arithmetic operations for any ϵ > 0. By analyzing the fourth power of the Coppersmith-Winograd tensor using our methods, we obtain the new lower bound α > 0.31389, which improves the previous lower bound α > 0.30298 obtained by Le Gall (FOCS'12) from the analysis of the second power of the Coppersmith-Winograd tensor. More generally, we give faster algorithms computing the product of an n × nk matrix by an nk × n matrix for any value k ≠ 1. (In the case k = 1, we recover the bounds recently obtained for square matrix multiplication). These improvements immediately lead to improvements in the complexity of a multitude of fundamental problems for which the bottleneck is rectangular matrix multiplication, such as computing the all-pair shortest paths in directed graphs with bounded weights.

High-dimensional asymptotics of prediction: Ridge regression and classification

Abstract: We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where $p,n\to\infty$ and $p/n\to\gamma>0$, and allow for arbitrary covariance among the features. For both methods, we provide an explicit and efficiently computable expression for the limiting predictive risk, which depends only on the spectrum of the feature-covariance matrix, the signal strength and the aspect ratio $\gamma$. Especially in the case of regularized discriminant analysis, we find that predictive accuracy has a nuanced dependence on the eigenvalue distribution of the covariance matrix, suggesting that analyses based on the operator norm of the covariance matrix may not be sharp. Our results also uncover an exact inverse relation between the limiting predictive risk and the limiting estimation risk in high-dimensional linear models. The analysis builds on recent advances in random matrix theory.

On the Capacity of Secure Distributed Matrix Multiplication

Abstract: Matrix multiplication is one of the key operations in various engineering applications. Outsourcing large-scale matrix multiplication tasks to multiple distributed servers or cloud is desirable to speed up computation. However, security becomes an issue when these servers are untrustworthy. In this paper, we study the problem of secure distributed matrix multiplication from distributed untrustworthy servers. This problem falls in the category of secure function computation and has received significant attention in the cryptography community. However, characterizing the fundamental limits of information-theoretically secure matrix multiplication remain an open problem. We focus on information-theoretically secure distributed matrix multiplication with the goal of characterizing the minimum communication overhead. The capacity of secure matrix multiplication is defined as the maximum possible ratio of the desired information and the total communication received from $N$ distributed servers. In particular, we study the following two models where we want to multiply two matrices $A\in \mathbb{F}^{m\times \mathrm{n}}$ and $B\in \mathbb{F}^{n\times p}$ : (a) one-sided secure matrix multiplication with $\ell$ colluding servers, in which $B$ is a public matrix available at all servers and $A$ is a private matrix. (b) fully secure matrix multiplication with $\ell$ colluding servers, in which both $A$ and $B$ are private matrices. The goal is to securely multiply $A$ and $B$ when any $\ell$ servers can collude. For model (a), we characterize the capacity as $C_{\mathbf{one}-\mathbf{sided}}^{(\ell)}= (N-\ell)/N$ by providing a secure matrix multiplication scheme and a matching converse. For model (b), we propose a novel scheme that lower bounds the capacity, i.e., $C_{\mathrm{fuly}}\geq(\lceil\sqrt{N}-\ell\rceil)^{2}/(\lceil\sqrt{N}-\ell\rceil+\ell)^{2}$ .

Recent Developments of Useful MALDI Matrices for the Mass Spectrometric Characterization of Lipids.

Abstract: Matrix-assisted laser desorption/ionization (MALDI) is one of the most successful "soft" ionization methods in the field of mass spectrometry and enables the analysis of a broad range of molecules, including lipids. Although the details of the ionization process are still unknown, the importance of the matrix is commonly accepted. Both, the development of and the search for useful matrices was, and still is, an empirical process, since properties like vacuum stability, high absorption at the laser wavelength, etc. have to be fulfilled by a compound to become a useful matrix. This review provides a survey of successfully used MALDI matrices for the lipid analyses of complex biological samples. The advantages and drawbacks of the established organic matrix molecules (cinnamic or benzoic acid derivatives), liquid crystalline matrices, and mixtures of common matrices will be discussed. Furthermore, we will deal with nanocrystalline matrices, which are most suitable to analyze small molecules, such as free fatty acids. It will be shown that the analysis of mixtures and the quantitative analysis of small molecules can be easily performed if the matrix is carefully selected. Finally, some basic principles of how useful matrix compounds can be "designed" de novo will be introduced.

One-loop Matching and Running with Covariant Derivative Expansion

Abstract: We develop tools for performing effective field theory (EFT) calculations in a manifestly gauge-covariant fashion. We clarify how functional methods account for one-loop diagrams resulting from the exchange of both heavy and light fields, as some confusion has recently arisen in the literature. To efficiently evaluate functional traces containing these “mixed” one-loop terms, we develop a new covariant derivative expansion (CDE) technique that is capable of evaluating a much wider class of traces than previous methods. The technique is detailed in an appendix, so that it can be read independently from the rest of this work. We review the well-known matching procedure to one-loop order with functional methods. What we add to this story is showing how to isolate one-loop terms coming from diagrams involving only heavy propagators from diagrams with mixed heavy and light propagators. This is done using a non-local effective action, which physically connects to the notion of “integrating out” heavy fields. Lastly, we show how to use a CDE to do running analyses in EFTs, i.e. to obtain the anomalous dimension matrix. We demonstrate the methodologies by several explicit example calculations.

Non-Convex Phase Retrieval From STFT Measurements

Abstract: The problem of recovering a one-dimensional signal from its Fourier transform magnitude, called Fourier phase retrieval, is ill-posed in most cases. We consider the closely-related problem of recovering a signal from its phaseless short-time Fourier transform (STFT) measurements. This problem arises naturally in several applications, such as ultra-short laser pulse characterization and ptychography. The redundancy offered by the STFT enables unique recovery under mild conditions. We show that in some cases the unique solution can be obtained by the principal eigenvector of a matrix, constructed as the solution of a simple least-squares problem. When these conditions are not met, we suggest using the principal eigenvector of this matrix to initialize non-convex local optimization algorithms and propose two such methods. The first is based on minimizing the empirical risk loss function, while the second maximizes a quadratic function on the manifold of phases. We prove that under appropriate conditions, the proposed initialization is close to the underlying signal. We then analyze the geometry of the empirical risk loss function and show numerically that both gradient algorithms converge to the underlying signal even with small redundancy in the measurements. In addition, the algorithms are robust to noise.

NeuTM: A neural network-based framework for traffic matrix prediction in SDN

Abstract: This paper presents NeuTM, a framework for network Traffic Matrix (TM) prediction based on Long Short-Term Memory Recurrent Neural Networks (LSTM RNNs). TM prediction is defined as the problem of estimating future network traffic matrix from the previous and achieved network traffic data. It is widely used in network planning, resource management and network security. Long Short-Term Memory (LSTM) is a specific recurrent neural network (RNN) architecture that is well-suited to learn from data and classify or predict time series with time lags of unknown size. LSTMs have been shown to model longrange dependencies more accurately than conventional RNNs. NeuTM is a LSTM RNN-based framework for predicting TM in large networks. By validating our framework on real-world data from GEANT network, we show that our model converges quickly and gives state of the art TM prediction performance.

Ab initio calculation of the shift photocurrent by Wannier interpolation

Abstract: We describe and implement a first-principles algorithm based on maximally localized Wannier functions for calculating the shift-current response of piezoelectric crystals in the independent-particle approximation. The proposed algorithm presents several advantages over existing ones, including full gauge invariance, low computational cost, and a correct treatment of the optical matrix elements with nonlocal pseudopotentials. Band-truncation errors are avoided by a careful formulation of $k\ifmmode\cdot\else\textperiodcentered\fi{}p$ perturbation theory within the subspace of wannierized bands. The needed ingredients are the matrix elements of the Hamiltonian and of the position operator in the Wannier basis, which are readily available at the end of the wannierization step. If the off-diagonal matrix elements of the position operator are discarded, our expressions reduce to the ones that have been used in recent tight-binding calculations of the shift current. We find that this ``diagonal'' approximation can introduce sizable errors, highlighting the importance of carefully embedding the tight-binding model in real space for an accurate description of the charge transfer that gives rise to the shift current.

FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network

Abstract: This paper develops the FastRNN and FastGRNN algorithms to address the twin RNN limitations of inaccurate training and inefficient prediction. Previous approaches have improved accuracy at the expense of prediction costs making them infeasible for resource-constrained and real-time applications. Unitary RNNs have increased accuracy somewhat by restricting the range of the state transition matrix's singular values but have also increased the model size as they require a larger number of hidden units to make up for the loss in expressive power. Gated RNNs have obtained state-of-the-art accuracies by adding extra parameters thereby resulting in even larger models. FastRNN addresses these limitations by adding a residual connection that does not constrain the range of the singular values explicitly and has only two extra scalar parameters. FastGRNN then extends the residual connection to a gate by reusing the RNN matrices to match state-of-the-art gated RNN accuracies but with a 2-4x smaller model. Enforcing FastGRNN's matrices to be low-rank, sparse and quantized resulted in accurate models that could be up to 35x smaller than leading gated and unitary RNNs. This allowed FastGRNN to accurately recognize the "Hey Cortana" wakeword with a 1 KB model and to be deployed on severely resource-constrained IoT microcontrollers too tiny to store other RNN models. FastGRNN's code is available at (https://github.com/Microsoft/EdgeML/).

A DEMATEL-based completion method for incomplete pairwise comparison matrix in AHP

Abstract: Pairwise comparison matrix (PCM) as a crucial component of Analytic Hierarchy Process (AHP) presents the preference relations among alternatives. However, in many cases, the PCM is difficult to be completed, which obstructs the subsequent operations of the classical AHP. In this paper, based on decision-making and trial evaluation laboratory (DEMATEL) method which has ability to derive the total relation matrix from direct relation matrix, a new completion method for incomplete pairwise comparison matrix (iPCM) is proposed. The proposed method provides a new perspective to estimate the missing values in iPCMs with explicit physical meaning, which is straightforward and flexible. Several experiments are implemented as well to present the completion ability of the proposed method and some insights into the proposed method and matrix consistency.

Guaranteed‐cost consensus for multiagent networks with Lipschitz nonlinear dynamics and switching topologies

Abstract: Guaranteed-cost consensus for high-order nonlinear multi-agent networks with switching topologies is investigated. By constructing a time-varying nonsingular matrix with a specific structure, the whole dynamics of multi-agent networks is decomposed into the consensus and disagreement parts with nonlinear terms, which is the key challenge to be dealt with. An explicit expression of the consensus dynamics, which contains the nonlinear term, is given and its initial state is determined. Furthermore, by the structure property of the time-varying nonsingular transformation matrix and the Lipschitz condition, the impacts of the nonlinear term on the disagreement dynamics are linearized and the gain matrix of the consensus protocol is determined on the basis of the Riccati equation. Moreover, an approach to minimize the guaranteed cost is given in terms of linear matrix inequalities. Finally, the numerical simulation is shown to demonstrate the effectiveness of theoretical results.