Showing papers on "Circulant matrix published in 2018"

Multi-modal Circulant Fusion for Video-to-Language and Backward.

Abstract: Multi-modal fusion has been widely involved in focuses of the modern artificial intelligence research, e.g., from visual content to languages and backward. Common-used multi-modal fusion methods mainly include element-wise product, element-wise sum, or even simply concatenation between different types of features, which are somewhat straightforward but lack in-depth analysis. Recent studies have shown fully exploiting interactions among elements of multi-modal features will lead to a further performance gain. In this paper, we put forward a new approach of multi-modal fusion, namely Multi-modal Circulant Fusion (MCF). Particularly, after reshaping feature vectors into circulant matrices, we define two types of interaction operations between vectors and matrices. As each row of the circulant matrix shifts one elements, with newly-defined interaction operations, we almost explore all possible interactions between vectors of different modalities. Moreover, as only regular operations are involved and defined a priori, MCF avoids increasing parameters or computational costs for multi-modal fusion. We evaluate MCF with tasks of video captioning and temporal activity localization via language (TALL). Experiments on MSVD and MSRVTT show our method obtains the state-of-the-art for video captioning. For TALL, by plugging into MCF, we achieve a performance gain of roughly 4.2% on TACoS.

Preconditioning and Iterative Solution of All-at-Once Systems for Evolutionary Partial Differential Equations

Abstract: Standard Krylov subspace solvers for self-adjoint problems have rigorous convergence bounds based solely on eigenvalues. However, for non-self-adjoint problems, eigenvalues do not determine behavior even for widely used iterative methods. In this paper, we discuss time-dependent PDE problems, which are always non-self-adjoint. We propose a block circulant preconditioner for the all-at-once evolutionary PDE system which has block Toeplitz structure. Through reordering of variables to obtain a symmetric system, we are able to rigorously establish convergence bounds for MINRES which guarantee a number of iterations independent of the number of time-steps for the all-at-once system. If the spatial differential operators are simultaneously diagonalizable, we are able to quickly apply the preconditioner through use of a sine transform; and for those that are not, we are able to use an algebraic multigrid process to provide a good approximation. Results are presented for solution to both the heat and convection ...

Analysis of Circulant Embedding Methods for Sampling Stationary Random Fields

Abstract: A standard problem in uncertainty quantification and in computational statistics is the sampling of stationary Gaussian random fields with given covariance in a $d$-dimensional (physical) domain. I...

On self-dual double circulant codes

Abstract: Self-dual double circulant codes of odd dimension are shown to be dihedral in even characteristic and consta-dihedral in odd characteristic. Exact counting formulae are derived for them, generalizing some old results of MacWilliams on the enumeration of circulant orthogonal matrices. These formulae, in turn, are instrumental in deriving a Varshamov–Gilbert bound on the relative minimum distance of this family of codes.

Circulant embedding with QMC: analysis for elliptic PDE with lognormal coefficients.

Abstract: In a previous paper (Graham et al. in J Comput Phys 230:3668–3694, 2011), the authors proposed a new practical method for computing expected values of functionals of solutions for certain classes of elliptic partial differential equations with random coefficients. This method was based on combining quasi-Monte Carlo (QMC) methods for computing the expected values with circulant embedding methods for sampling the random field on a regular grid. It was found capable of handling fluid flow problems in random heterogeneous media with high stochastic dimension, but no convergence theory was provided. This paper provides a convergence analysis for the method in the case when the QMC method is a specially designed randomly shifted lattice rule. The convergence result depends on the eigenvalues of the underlying nested block circulant matrix and can be independent of the number of stochastic variables under certain assumptions. In fact the QMC analysis applies to general factorisations of the covariance matrix to sample the random field. The error analysis for the underlying fully discrete finite element method allows for locally refined meshes (via interpolation from a regular sampling grid of the random field). Numerical results on a non-regular domain with corner singularities in two spatial dimensions and on a regular domain in three spatial dimensions are included.

Identification of Sparse Reciprocal Graphical Models

Abstract: In this letter we propose an identification procedure of a sparse graphical model associated to a Gaussian stationary stochastic process. The identification paradigm exploits the approximation of autoregressive (AR) processes through reciprocal processes in order to improve the robustness of the identification algorithm, especially when the order of the AR process becomes large. We show that the proposed paradigm leads to a regularized, circulant matrix completion problem whose solution only requires computations of the eigenvalues of matrices of dimension equal to the dimension of the process.

The tensor t-function: a definition for functions of third-order tensors

Abstract: A definition for functions of multidimensional arrays is presented. The definition is valid for third-order tensors in the tensor t-product formalism, which regards third-order tensors as block circulant matrices. The tensor function definition is shown to have similar properties as standard matrix function definitions in fundamental scenarios. To demonstrate the definition's potential in applications, the notion of network communicability is generalized to third-order tensors and computed for a small-scale example via block Krylov subspace methods for matrix functions. A complexity analysis for these methods in the context of tensors is also provided.

Sparse Orthogonal Circulant Transform Multiplexing for Coherent Optical Fiber Communication

Abstract: This paper introduces a new multicarrier system, named sparse orthogonal circulant transform multiplexing (S-OCTM), for optical fiber communication This technique uses an inverse sparse orthogonal circulant transform (S-OCT) matrix, which is simple and contains only two nonzero elements in each column, to multiplex information of different subcarriers We compared the proposed scheme with conventional orthogonal frequency division multiplexing (OFDM), orthogonal chirp division multiplexing (OCDM), and discrete-Fourier-transform spreading OFDM (DFT-S-OFDM) in a coherent optical communication system It is shown that S-OCTM, while exhibiting the complexity among the least, avoids the performance disadvantages of all investigated conventional schemes It is theoretically proved that the S-OCT matrix equalizes the bandwidth limitation effect that degrades the performance of conventional OFDM It also shows a greatly reduced peak-to-average power ratio and higher tolerance to fiber nonlinearity than OFDM and OCDM On the other hand, compared to DFT-S-OFDM, S-OCTM shows a better dispersion tolerance under insufficient length of cyclic prefix and is more tolerable to strong optical filtering The performance advantages and low complexity enable the proposed scheme to be a promising multicarrier solution for optical communications

Quantum circulant preconditioner for a linear system of equations

Abstract: We consider the quantum linear solver for $Ax=b$ with the circulant preconditioner $C$. The main technique is the singular value estimation (SVE) introduced in [Kerenidis and Prakash, Quantum recommendation system, in ITCS (2017)]. However, the SVE should be modified to solve the preconditioned linear system ${C}^{\ensuremath{-}1}Ax={C}^{\ensuremath{-}1}b$. Moreover, different from the preconditioned linear system considered in [Phys. Rev. Lett. 110, 250504 (2013)], the circulant preconditioner is easy to construct and can be directly applied to general dense non-Hermitian cases. The time complexity depends on the condition numbers of $C$ and ${C}^{\ensuremath{-}1}A$, as well as the Frobenius norm ${\ensuremath{\parallel}A\ensuremath{\parallel}}_{F}$.

Representations of Gaussian Random Fields and Approximation of Elliptic PDEs with Lognormal Coefficients

Abstract: Approximation of elliptic PDEs with random diffusion coefficients typically requires a representation of the diffusion field in terms of a sequence \(y=(y_j)_{j\ge 1}\) of scalar random variables. One may then apply high-dimensional approximation methods to the solution map \(y\mapsto u(y)\). Although Karhunen–Loeve representations are commonly used, it was recently shown, in the relevant case of lognormal diffusion fields, that multilevel-type expansions may yield better approximation rates. Motivated by these results, we construct wavelet-type representations of stationary Gaussian random fields defined on arbitrary bounded domains. The size and localization properties of these wavelets are studied, and used to obtain polynomial approximation results for the related elliptic PDE which outperform those achievable when using Karhunen–Loeve representations. Our construction is based on a periodic extension of the stationary random field, and the expansion on the domain is then obtained by simple restriction. This makes the approach easily applicable even when the computational domain of the PDE has a complicated geometry. In particular, we apply this construction to the class of Gaussian processes defined by the family of Matern covariances. The proposed periodic continuation technique has other relevant applications such as fast simulation of trajectories. It can be regarded as a continuous analog of circulant embedding techniques introduced for Toeplitz matrices. One of its specific features is that the rate of decay of the eigenvalues of the covariance operator of the periodized process provably matches that of the Fourier transform of the covariance function of the original process.

Efficient Modeling of Gravity Fields Caused by Sources with Arbitrary Geometry and Arbitrary Density Distribution

Abstract: We present a brief review of gravity forward algorithms in Cartesian coordinate system, including both space-domain and Fourier-domain approaches, after which we introduce a truly general and efficient algorithm, namely the convolution-type Gauss fast Fourier transform (Conv-Gauss-FFT) algorithm, for 2D and 3D modeling of gravity potential and its derivatives due to sources with arbitrary geometry and arbitrary density distribution which are defined either by discrete or by continuous functions. The Conv-Gauss-FFT algorithm is based on the combined use of a hybrid rectangle-Gaussian grid and the fast Fourier transform (FFT) algorithm. Since the gravity forward problem in Cartesian coordinate system can be expressed as continuous convolution-type integrals, we first approximate the continuous convolution by a weighted sum of a series of shifted discrete convolutions, and then each shifted discrete convolution, which is essentially a Toeplitz system, is calculated efficiently and accurately by combining circulant embedding with the FFT algorithm. Synthetic and real model tests show that the Conv-Gauss-FFT algorithm can obtain high-precision forward results very efficiently for almost any practical model, and it works especially well for complex 3D models when gravity fields on large 3D regular grids are needed.

Fast random field generation with H -matrices

Abstract: We use the H-matrix technology to compute the approximate square root of a covariance matrix in linear cost. This allows us to generate normal and log-normal random fields on general point sets with optimal cost. We derive rigorous error estimates which show convergence of the method. Our approach requires only mild assumptions on the covariance function and on the point set. Therefore, it might be also a nice alternative to the circulant embedding approach which applies only to regular grids and stationary covariance functions.

Improved bounds for sparse recovery from subsampled random convolutions

Abstract: We study the recovery of sparse vectors from subsampled random convolutions via $\ell_{1}$-minimization. We consider the setup in which both the subsampling locations as well as the generating vector are chosen at random. For a sub-Gaussian generator with independent entries, we improve previously known estimates: if the sparsity $s$ is small enough, that is, $s\lesssim\sqrt{n/\log(n)}$, we show that $m\gtrsim s\log(en/s)$ measurements are sufficient to recover $s$-sparse vectors in dimension $n$ with high probability, matching the well-known condition for recovery from standard Gaussian measurements. If $s$ is larger, then essentially $m\geq s\log^{2}(s)\log(\log(s))\log(n)$ measurements are sufficient, again improving over previous estimates. Our results are shown via the so-called robust null space property which is weaker than the standard restricted isometry property. Our method of proof involves a novel combination of small ball estimates with chaining techniques which should be of independent interest.

Automorphism groups of circulant digraphs with applications to semigroup theory

Abstract: We characterize the automorphism groups of circulant digraphs whose connection sets are relatively small, and of unit circulant digraphs. For each class, we either explicitly determine the automorphism group or we show that the graph is a “normal” circulant, so the automorphism group is contained in the normalizer of a cycle. Then we use these characterizations to prove results on the automorphisms of the endomorphism monoids of those digraphs. The paper ends with a list of open problems on graphs, number theory, groups and semigroups.

Limited Memory Block Preconditioners for Fast Solution of Fractional Partial Differential Equations

Abstract: An innovative block structured with sparse blocks multi iterative preconditioner for linear multistep formulas used in boundary value form is proposed here to accelerate GMRES, FGMRES and BiCGstab(l). The preconditioner is based on block $$\omega $$ -circulant matrices and a short-memory approximation of the underlying Jacobian matrix of the fractional partial differential equations. Convergence results, numerical tests and comparisons with other techniques confirm the effectiveness of the approach.

Fast Structured Matrix Computations: Tensor Rank and Cohn---Umans Method

Abstract: We discuss a generalization of the Cohn---Umans method, a potent technique developed for studying the bilinear complexity of matrix multiplication by embedding matrices into an appropriate group algebra. We investigate how the Cohn---Umans method may be used for bilinear operations other than matrix multiplication, with algebras other than group algebras, and we relate it to Strassen's tensor rank approach, the traditional framework for investigating bilinear complexity. To demonstrate the utility of the generalized method, we apply it to find the fastest algorithms for forming structured matrix---vector product, the basic operation underlying iterative algorithms for structured matrices. The structures we study include Toeplitz, Hankel, circulant, symmetric, skew-symmetric, f-circulant, block Toeplitz---Toeplitz block, triangular Toeplitz matrices, Toeplitz-plus-Hankel, sparse/banded/triangular. Except for the case of skew-symmetric matrices, for which we have only upper bounds, the algorithms derived using the generalized Cohn---Umans method in all other instances are the fastest possible in the sense of having minimum bilinear complexity. We also apply this framework to a few other bilinear operations including matrix---matrix, commutator, simultaneous matrix products, and briefly discuss the relation between tensor nuclear norm and numerical stability.

Robust one-bit compressed sensing with partial circulant matrices.

Abstract: We present optimal sample complexity estimates for one-bit compressed sensing problems in a realistic scenario: the procedure uses a structured matrix (a randomly sub-sampled circulant matrix) and is robust to analog pre-quantization noise as well as to adversarial bit corruptions in the quantization process. Our results imply that quantization is not a statistically expensive procedure in the presence of nontrivial analog noise: recovery requires the same sample size one would have needed had the measurement matrix been Gaussian and the noisy analog measurements been given as data.

Tensor Completion Based on Triple Tubal Nuclear Norm

Abstract: Many tasks in computer vision suffer from missing values in tensor data, i.e., multi-way data array. The recently proposed tensor tubal nuclear norm (TNN) has shown superiority in imputing missing values in 3D visual data, like color images and videos. However, by interpreting in a circulant way, TNN only exploits tube (often carrying temporal/channel information) redundancy in a circulant way while preserving the row and column (often carrying spatial information) relationship. In this paper, a new tensor norm named the triple tubal nuclear norm (TriTNN) is proposed to simultaneously exploit tube, row and column redundancy in a circulant way by using a weighted sum of three TNNs. Thus, more spatial-temporal information can be mined. Further, a TriTNN-based tensor completion model with an ADMM solver is developed. Experiments on color images, videos and LiDAR datasets show the superiority of the proposed TriTNN against state-of-the-art nuclear norm-based tensor norms.

Asymptotics and Arithmetical Properties of Complexity for Circulant Graphs

Abstract: —We study analytical and arithmetical properties of the complexity function for infinite families of circulant C n (s1, s2,…, s k ) C2n(s1, s2,…, s k , n). Exact analytical formulas for the complexity functions of these families are derived, and their asymptotics are found. As a consequence, we show that the thermodynamic limit of these families of graphs coincides with the small Mahler measure of the accompanying Laurent polynomials.

Building Efficient Deep Neural Networks with Unitary Group Convolutions

Abstract: We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and block-circulant networks (i.e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique. We experimentally demonstrate that dense unitary transforms can outperform channel shuffling in DNN accuracy. On the other hand, different dense transforms exhibit comparable accuracy performance. Based on these observations we propose HadaNet, a UGConv network using Hadamard transforms. HadaNets achieve similar accuracy to circulant networks with lower computation complexity, and better accuracy than ShuffleNets with the same number of parameters and floating-point multiplies.

Development of a Universal Adaptive Fast Algorithm for the Synthesis of Circulant Topologies for Networks-on-Chip Implementations

Abstract: In this article, the feasibility of realization of optimal circulant topologies in networks-on-chip was researched. The software for automating the synthesis of circulant topologies of various dimensions and of any number of generatrices is presented. The implemented methods to speed up the synthesis process, based on the properties of circulants, as well as improving the algorithm for calculation of the distance between nodes and caching the adjacency matrix to achieve an acceptable search speed, are proposed. The efficiency and correctness of the proposed algorithm were verified. The proposed algorithm and methods allow designing networks-on-chip with improved characteristics of diameter, average distance between nodes, edges count, and, as a result, reducing the area, occupied by network-on-chip, and other characteristics, in comparison with analogues based on other widespread regular topologies.

Respectively scaled HSS iteration methods for solving discretized spatial fractional diffusion equations

Abstract: Summary For the discrete linear systems resulted from the discretization of the one-dimensional anisotropic spatial fractional diffusion equations of variable coefficients with the shifted finite-difference formulas of the Grunwald–Letnikov type, we propose a class of respectively scaled Hermitian and skew-Hermitian splitting iteration method and establish its asymptotic convergence theory. The corresponding induced matrix splitting preconditioner, through further replacements of the involved Toeplitz matrices with certain circulant matrices, leads to an economic variant that can be executed by fast Fourier transforms. Both theoretical analysis and numerical implementations show that this fast respectively scaled Hermitian and skew-Hermitian splitting preconditioner can significantly improve the computational efficiency of the Krylov subspace iteration methods employed as effective linear solvers for the target discrete linear systems.

Symmetric Pseudo-Random Matrices

Abstract: We consider the problem of generating symmetric pseudo-random sign (±1) matrices based on the similarity of their spectra to Wigner’s semicircular law. Using binary $m$ -sequences (Golomb sequences) of lengths $n=2^{m}-1$ , we give a simple explicit construction of circulant $n \times n$ sign matrices and show that their spectra converge to the semicircular law when $n$ grows. The Kolmogorov complexity of the proposed matrices equals to that of Golomb sequences and is at most $2 {\mathrm{log}}_{2}(n)$ bits.

Randomized Kernel Selection With Spectra of Multilevel Circulant Matrices.

Abstract: Kernel selection aims at choosing an appropriate kernel function for kernel-based learning algorithms to avoid either underfitting or overfitting of the resulting hypothesis. One of the main problems faced by kernel selection is the evaluation of the goodness of a kernel, which is typically difficult and computationally expensive. In this paper, we propose a randomized kernel selection approach to evaluate and select the kernel with the spectra of the specifically designed multilevel circulant matrices (MCMs), which is statistically sound and computationally efficient. Instead of constructing the kernel matrix, we construct the randomized MCM to encode the kernel function and all data points together with labels. We build a one-to-one correspondence between all candidate kernel functions and the spectra of the randomized MCMs by Fourier transform. We prove the statistical properties of the randomized MCMs and the randomized kernel selection criteria, which theoretically qualify the utility of the randomized criteria in kernel selection. With the spectra of the randomized MCMs, we derive a series of randomized criteria to conduct kernel selection, which can be computed in log-linear time and linear space complexity by fast Fourier transform (FFT). Experimental results demonstrate that our randomized kernel selection criteria are significantly more efficient than the existing classic and widely-used criteria while preserving similar predictive performance.

A fast implicit difference scheme for a new class of time distributed-order and space fractional diffusion equations with variable coefficients

Abstract: Recently, several problems in mathematics, physics, and engineering have been modeled via distributed-order fractional diffusion equations. In this paper, a new class of time distributed-order and space fractional diffusion equations with variable coefficients on bounded domains and Dirichlet boundary conditions is considered. By performing numerical integration we transform the time distributed-order fractional diffusion equations into multiterm time-space fractional diffusion equations. An implicit difference scheme for the multiterm time-space fractional diffusion equations is proposed along with a discussion about the unconditional stability and convergence. Then, the fast Krylov subspace methods with suitable circulant preconditioners are developed to solve the resultant linear system in light of their Toeplitz-like structures. The aforementioned methods are proved to acquire the capability to reduce the memory storage of the proposed implicit difference scheme from $\mathcal{O}(M^{2})$ to $\mathcal {O}(M)$ and the computational cost from $\mathcal{O}(M^{3})$ to $\mathcal{O}(M\log M)$ during iteration procedures, where M is the number of grid nodes. Finally, numerical experiments are employed to support the theoretical findings and show the efficiency of the proposed methods.