Showing papers on "Sparse matrix published in 2021"

A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method

Abstract: Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. A momentum method is frequently adopted to accelerate a learning algorithm, but it is incompatible with those implicitly adopting gradients like SLF-NMU. To build a fast NLF (FNLF) model, we propose a generalized momentum method compatible with SLF-NMU. With it, we further propose a single latent factor-dependent non-negative, multiplicative and momentum-incorporated update algorithm, thereby achieving an FNLF model. Empirical studies on six HiDS matrices from industrial application indicate that an FNLF model outperforms an NLF model in terms of both convergence rate and prediction accuracy for missing data. Hence, compared with an NLF model, an FNLF model is more practical in industrial applications.

Robust Low-Rank Tensor Recovery with Rectification and Alignment

Abstract: Low-rank tensor recovery in the presence of sparse but arbitrary errors is an important problem with many practical applications. In this work, we propose a general framework that recovers low-rank tensors, in which the data can be deformed by some unknown transformations and corrupted by arbitrary sparse errors. We give a unified presentation of the surrogate-based formulations that incorporate the features of rectification and alignment simultaneously, and establish worst-case error bounds of the recovered tensor. In this context, the state-of-the-art methods ‘RASL’ and ‘TILT’ can be viewed as two special cases of our work, and yet each only performs part of the function of our method. Subsequently, we study the optimization aspects of the problem in detail by deriving two algorithms, one based on the alternating direction method of multipliers (ADMM) and the other based on proximal gradient. We provide convergence guarantees for the latter algorithm, and demonstrate the performance of the former through in-depth simulations. Finally, we present extensive experimental results on public datasets to demonstrate the effectiveness and efficiency of the proposed framework and algorithms.

A Deep Latent Factor Model for High-Dimensional and Sparse Matrices in Recommender Systems

Abstract: Recommender systems (RSs) commonly adopt a user-item rating matrix to describe users’ preferences on items. With users and items exploding, such a matrix is usually high-dimensional and sparse (HiDS). Recently, the idea of deep learning has been applied to RSs. However, current deep-structured RSs suffer from high computational complexity. Enlightened by the idea of deep forest, this paper proposes a deep latent factor model (DLFM) for building a deep-structured RS on an HiDS matrix efficiently. Its main idea is to construct a deep-structured model by sequentially connecting multiple latent factor (LF) models instead of multilayered neural networks through a nonlinear activation function. Thus, the computational complexity grows linearly with its layer count, which is easy to resolve in practice. The experimental results on four HiDS matrices from industrial RSs demonstrate that when compared with state-of-the-art LF models and deep-structured RSs, DLFM can well balance the prediction accuracy and computational efficiency, which well fits the desire of industrial RSs for fast and right recommendations.

Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution

Abstract: Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named “structured sparse low-rank representation” (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.

An Instance-Frequency-Weighted Regularization Scheme for Non-Negative Latent Factor Analysis on High-Dimensional and Sparse Data

Abstract: High-dimensional and sparse (HiDS) data with non-negativity constraints are commonly seen in industrial applications, such as recommender systems. They can be modeled into an HiDS matrix, from which non-negative latent factor analysis (NLFA) is highly effective in extracting useful features. Preforming NLFA on an HiDS matrix is ill-posed, desiring an effective regularization scheme for avoiding overfitting. Current models mostly adopt a standard ${L} _{2}$ scheme, which does not consider the imbalanced distribution of known data in an HiDS matrix. From this point of view, this paper proposes an instance-frequency-weighted regularization (IR) scheme for NLFA on HiDS data. It specifies the regularization effects on each latent factors with its relevant instance count, i.e., instance-frequency, which clearly describes the known data distribution of an HiDS matrix. By doing so, it achieves finely grained modeling of regularization effects. The experimental results on HiDS matrices from industrial applications demonstrate that compared with an ${L} _{2}$ scheme, an IR scheme enables a resultant model to achieve higher accuracy in missing data estimation of an HiDS matrix.

Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data

Abstract: High-dimensional and sparse (HiDS) matrices commonly arise in various industrial applications, e.g., recommender systems (RSs), social networks, and wireless sensor networks. Since they contain rich information, how to accurately represent them is of great significance. A latent factor (LF) model is one of the most popular and successful ways to address this issue. Current LF models mostly adopt $L_{2}$ -norm-oriented Loss to represent an HiDS matrix, i.e., they sum the errors between observed data and predicted ones with $L_{2}$ -norm. Yet $L_{2}$ -norm is sensitive to outlier data. Unfortunately, outlier data usually exist in such matrices. For example, an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users. To address this issue, this work proposes a smooth $L_{1}$ -norm-oriented latent factor (SL-LF) model. Its main idea is to adopt smooth $L_{1}$ -norm rather than $L_{2}$ -norm to form its Loss, making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix. Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.

AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing

Abstract: Most compressive sensing (CS) reconstruction methods can be divided into two categories, i.e. model-based methods and classical deep network methods. By unfolding the iterative optimization algorithm for model-based methods onto networks, deep unfolding methods have the good interpretation of model-based methods and the high speed of classical deep network methods. In this article, to solve the visual image CS problem, we propose a deep unfolding model dubbed AMP-Net. Rather than learning regularization terms, it is established by unfolding the iterative denoising process of the well-known approximate message passing algorithm. Furthermore, AMP-Net integrates deblocking modules in order to eliminate the blocking artifacts that usually appear in CS of visual images. In addition, the sampling matrix is jointly trained with other network parameters to enhance the reconstruction performance. Experimental results show that the proposed AMP-Net has better reconstruction accuracy than other state-of-the-art methods with high reconstruction speed and a small number of network parameters.

Multi-UAV Cooperative Localization for Marine Targets Based on Weighted Subspace Fitting in SAGIN Environment

Abstract: As an indispensable part of Internet of Vehicles (IoV), unmanned aerial vehicles (UAVs) can be deployed for target positioning and navigation in space-air-ground integrated networks (SAGIN) environment Maritime target positioning is very important for the safe navigation of ships, hydrographic surveys, and marine resource exploration Traditional methods typically exploit satellites to locate marine targets in SAGIN environment, and the location accuracy does not satisfy the requirements of modern ocean observation missions In order to localize marine target, we develop a system architecture in this paper, which contains UAVs integrated with monostatic multiple-input multiple-output (MIMO) radars The main thrust is to estimate direction-of-arrival (DOA) via MIMO radar Herein, we consider a general scenario that unknown mutual coupling exist, and a novel sparse reconstruction algorithm is proposed The mutual coupling matrix (MCM) is adopted with the help of its special structure, we formulate the data model as a sparse representation form Then two novel matrices, a weighted matrix and a reduced-dimensional matrix are constructed to reduce the computational complexity and enhance the sparsity, respectively Thereafter, a sparse constraint model is constructed using the concept of optimal weighted subspace fitting (WSF) Finally, DOA estimation of maritime targets can be achieved by reconstructing the support of a block sparse matrix Based on the DOA estimation results, multiple UAVs are used to cross-locate marine targets multiple times, and an accurate marine target position is achieved in SAGIN environment Numerical results are carried out, which demonstrates the effectiveness of the proposed DOA estimator, and the multi-UAV cooperative localization system can realize accurate target localization

Robust Design for Intelligent Reflecting Surface-Assisted MIMO-OFDMA Terahertz IoT Networks

Abstract: Terahertz (THz) communication has been regarded as one promising technology to enhance the transmission capacity of future Internet-of-Things (IoT) users due to its ultrawide bandwidth. Nonetheless, one major obstacle that prevents the actual deployment of THz lies in its inherent huge attenuation. Intelligent reflecting surface (IRS) and multiple-input–multiple-output (MIMO) represent two effective solutions for compensating the large path loss in THz systems. In this article, we consider an IRS-aided multiuser THz MIMO system with orthogonal frequency-division multiple (OFDM) access, where the sparse radio frequency chain antenna structure is adopted for reducing the power consumption. The objective is to maximize the weighted sum rate via jointly optimizing the hybrid analog/digital beamforming at the base station (BS) and reflection matrix at the IRS. Since the analog beamforming and reflection matrix need to cater all users and subcarriers, it is difficult to directly solve the formulated problem, and thus, an alternatively iterative optimization algorithm is proposed. Specifically, the analog beamforming is designed by solving a MIMO capacity maximization problem, while the digital beamforming and reflection matrix optimization are both tackled using semidefinite relaxation (SDR) technique. Considering that obtaining perfect channel state information (CSI) is a challenging task in IRS-based systems, we further explore the case with the imperfect CSI for the channels from the IRS to users. Under this setup, we propose a robust beamforming and reflection matrix design scheme for the originally formulated nonconvex optimization problem. Finally, simulation results are presented to demonstrate the effectiveness of the proposed algorithms.

A Projective and Discriminative Dictionary Learning for High-Dimensional Process Monitoring With Industrial Applications

Abstract: Data-driven process monitoring methods have attracted many attentions and gained wide applications. However, the real industrial process data are much more complex which is characterized by multimode, high dimensional, corrupted, and less labeled data. In order to eliminate these unfavourable factors simultaneously, a semisupervised robust projective and discriminative dictionary learning method is proposed. First, a semisupervised strategy is introduced to label unsupervised training data. Then, by utilizing low-rank and sparse features of raw data and outliers, a robust decomposition method is used to obtain clean data. After that, a simultaneously projective and discriminative model is proposed to extracting the feature of the low-rank clean data. Finally, the projection matrix and global dictionary, as well as the threshold are obtained through iterative dictionary learning. This hybrid framework provides a robust model for process monitoring and mode identification, and its efficiency is demonstrated by both synthetic examples and real industrial process cases.

Determinant-Based Fast Greedy Sensor Selection Algorithm

Abstract: In this paper, the sparse sensor placement problem for least-squares estimation is considered, and the previous novel approach of the sparse sensor selection algorithm is extended. The maximization of the determinant of the matrix which appears in pseudo-inverse matrix operations is employed as an objective function of the problem in the present extended approach. The procedure for the maximization of the determinant of the corresponding matrix is proved to be mathematically the same as that of the previously proposed QR method when the number of sensors is less than that of state variables (undersampling). On the other hand, the authors have developed a new algorithm for when the number of sensors is greater than that of state variables (oversampling). Then, a unified formulation of the two algorithms is derived, and the lower bound of the objective function given by this algorithm is shown using the monotone submodularity of the objective function. The effectiveness of the proposed algorithm on the problem using real datasets is demonstrated by comparing with the results of other algorithms. The numerical results show that the proposed algorithm improves the estimation error by approximately 10% compared with the conventional methods in the oversampling case, where the estimation error is defined as the ratio of the difference between the reconstructed data and the full observation data to the full observation. For the NOAA-SST sensor problem, which has more than ten thousand sensor candidate points, the proposed algorithm selects the sensor positions in few seconds, which required several hours with the other algorithms in the oversampling case on a 3.40 GHz computer.

An α -β -Divergence-Generalized Recommender for Highly Accurate Predictions of Missing User Preferences

Abstract: To quantify user-item preferences, a recommender system (RS) commonly adopts a high-dimensional and sparse (HiDS) matrix. Such a matrix can be represented by a non-negative latent factor analysis model relying on a single latent factor (LF)-dependent, non-negative, and multiplicative update algorithm. However, existing models' representative abilities are limited due to their specialized learning objective. To address this issue, this study proposes an α-β-divergence-generalized model that enjoys fast convergence. Its ideas are three-fold: 1) generalizing its learning objective with α -β -divergence to achieve highly accurate representation of HiDS data; 2) incorporating a generalized momentum method into parameter learning for fast convergence; and 3) implementing self-adaptation of controllable hyperparameters for excellent practicability. Empirical studies on six HiDS matrices from real RSs demonstrate that compared with state-of-the-art LF models, the proposed one achieves significant accuracy and efficiency gain to estimate huge missing data in an HiDS matrix.

Hyperspectral and Multispectral Image Fusion via Graph Laplacian-Guided Coupled Tensor Decomposition

Abstract: We propose a novel graph Laplacian-guided coupled tensor decomposition (gLGCTD) model for fusion of hyperspectral image (HSI) and multispectral image (MSI) for spatial and spectral resolution enhancements. The coupled Tucker decomposition is employed to capture the global interdependencies across the different modes to fully exploit the intrinsic global spatial–spectral information. To preserve local characteristics, the complementary submanifold structures embedded in high-resolution (HR)-HSI are encoded by the graph Laplacian regularizations. The global spatial–spectral information captured by the coupled Tucker decomposition and the local submanifold structures are incorporated into a unified framework. The gLGCTD fusion framework is solved by a hybrid framework between the proximal alternating optimization (PAO) and the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real data sets demonstrate that the gLGCTD fusion method is superior to state-of-the-art fusion methods with a more accurate reconstruction of the HR-HSI.

Image Restoration via Reconciliation of Group Sparsity and Low-Rank Models

Abstract: Image nonlocal self-similarity (NSS) property has been widely exploited via various sparsity models such as joint sparsity (JS) and group sparse coding (GSC). However, the existing NSS-based sparsity models are either too restrictive, e.g. , JS enforces the sparse codes to share the same support, or too general, e.g. , GSC imposes only plain sparsity on the group coefficients, which limit their effectiveness for modeling real images. In this paper, we propose a novel NSS-based sparsity model, namely, low-rank regularized group sparse coding (LR-GSC) , to bridge the gap between the popular GSC and JS. The proposed LR-GSC model simultaneously exploits the sparsity and low-rankness of the dictionary-domain coefficients for each group of similar patches. An alternating minimization with an adaptive adjusted parameter strategy is developed to solve the proposed optimization problem for different image restoration tasks, including image denoising, image deblocking, image inpainting, and image compressive sensing. Extensive experimental results demonstrate that the proposed LR-GSC algorithm outperforms many popular or state-of-the-art methods in terms of objective and perceptual metrics.

Hyperspectral and Multispectral Image Fusion via Nonlocal Low-Rank Tensor Approximation and Sparse Representation

Abstract: The fusion of hyperspectral (HS) and multispectral (MS) images designed to obtain high-resolution HS (HRHS) images is a very challenging work. A series of solutions has been proposed in recent years. However, the similarity in the structure of the HS image has not been fully used. In this article, we present a novel HS and MS image-fusion method based on nonlocal low-rank tensor approximation and sparse representation. Specifically, the HS image and the MS image are considered the spatially and spectrally degraded versions of the HRHS image, respectively. Then, the nonlocal low-rank constraint term is adopted in order to form the nonlocal similarity and the spatial–spectral correlation. Meanwhile, we add the sparse constraint term to describe the sparsity of abundance. Thus, the proposed fusion model is established and its optimization is solved by alternative direction method of multipliers (ADMM). The experimental results on three synthetic data sets and one real data set show the advantages of the proposed method over several state-of-the-art competitors.

Learning Optical Flow from a Few Matches

Abstract: State-of-the-art neural network models for optical flow estimation require a dense correlation volume at high resolutions for representing per-pixel displacement. Although the dense correlation volume is informative for accurate estimation, its heavy computation and memory usage hinders the efficient training and deployment of the models. In this paper, we show that the dense correlation volume representation is redundant and accurate flow estimation can be achieved with only a fraction of elements in it. Based on this observation, we propose an alternative displacement representation, named Sparse Correlation Volume, which is constructed directly by computing the k closest matches in one feature map for each feature vector in the other feature map and stored in a sparse data structure. Experiments show that our method can reduce computational cost and memory use significantly, while maintaining high accuracy compared to previous approaches with dense correlation volumes.

BATCH: A Scalable Asymmetric Discrete Cross-Modal Hashing

Abstract: Supervised cross-modal hashing has attracted much attention. However, there are still some challenges, e.g., how to effectively embed the label information into binary codes, how to avoid using a large similarity matrix and make a model scalable to large-scale datasets, how to efficiently solve the binary optimization problem. To address these challenges, in this paper, we present a novel supervised cross-modal hashing method, i.e., scalaBle Asymmetric discreTe Cross-modal Hashing, BATCH for short. It leverages collective matrix factorization to learn a common latent space for the labels and different modalities, and embeds the labels into binary codes by minimizing a distance-distance difference problem. Furthermore, it builds a connection between the common latent space and the hash codes by an asymmetric strategy. In the light of this, it can perform cross-modal retrieval and embed more similarity information into the binary codes. In addition, it introduces a quantization minimization term and orthogonal constraints into the optimization problem, and generates the binary codes discretely. Therefore, the quantization error and redundancy may be much reduced. Moreover, it is a two-step method, making the optimization simple and scalable to large-scale datasets. Extensive experimental results on three benchmark datasets demonstrate that BATCH outperforms some state-of-the-art cross-modal hashing methods in terms of accuracy and efficiency.

Jointly Sparse Signal Recovery and Support Recovery via Deep Learning With Applications in MIMO-Based Grant-Free Random Access

Abstract: In this article, we investigate jointly sparse signal recovery and jointly sparse support recovery in Multiple Measurement Vector (MMV) models for complex signals, which arise in many applications in communications and signal processing. Recent key applications include channel estimation and device activity detection in MIMO-based grant-free random access which is proposed to support massive machine-type communications (mMTC) for Internet of Things (IoT). Utilizing techniques in compressive sensing, optimization and deep learning, we propose two model-driven approaches, based on the standard auto-encoder structure for real numbers. One is to jointly design the common measurement matrix and jointly sparse signal recovery method, and the other aims to jointly design the common measurement matrix and jointly sparse support recovery method. The proposed model-driven approaches can effectively utilize features of sparsity patterns in designing common measurement matrices and adjusting model-driven decoders, and can greatly benefit from the underlying state-of-the-art recovery methods with theoretical guarantee. Hence, the obtained common measurement matrices and recovery methods can significantly outperform the underlying advanced recovery methods. We conduct extensive numerical results on channel estimation and device activity detection in MIMO-based grant-free random access. The numerical results show that the proposed approaches provide pilot sequences and channel estimation or device activity detection methods which can achieve higher estimation or detection accuracy with shorter computation time than existing ones. Furthermore, the numerical results explain how such gains are achieved via the proposed approaches.

Semisupervised Adaptive Symmetric Non-Negative Matrix Factorization

Abstract: As a variant of non-negative matrix factorization (NMF), symmetric NMF (SymNMF) can generate the clustering result without additional post-processing, by decomposing a similarity matrix into the product of a clustering indicator matrix and its transpose. However, the similarity matrix in the traditional SymNMF methods is usually predefined, resulting in limited clustering performance. Considering that the quality of the similarity graph is crucial to the final clustering performance, we propose a new semisupervised model, which is able to simultaneously learn the similarity matrix with supervisory information and generate the clustering results, such that the mutual enhancement effect of the two tasks can produce better clustering performance. Our model fully utilizes the supervisory information in the form of pairwise constraints to propagate it for obtaining an informative similarity matrix. The proposed model is finally formulated as a non-negativity-constrained optimization problem. Also, we propose an iterative method to solve it with the convergence theoretically proven. Extensive experiments validate the superiority of the proposed model when compared with nine state-of-the-art NMF models.

Bayesian Low-Tubal-Rank Robust Tensor Factorization with Multi-Rank Determination

Abstract: Robust tensor factorization is a fundamental problem in machine learning and computer vision, which aims at decomposing tensors into low-rank and sparse components. However, existing methods either suffer from limited modeling power in preserving low-rank structures, or have difficulties in determining the target tensor rank and the trade-off between the low-rank and sparse components. To address these problems, we propose a fully Bayesian treatment of robust tensor factorization along with a generalized sparsity-inducing prior. By adapting the recently proposed low-tubal-rank model in a generative manner, our method is effective in preserving low-rank structures. Moreover, benefiting from the proposed prior and the Bayesian framework, the proposed method can automatically determine the tensor rank while inferring the trade-off between the low-rank and sparse components. For model estimation, we develop a variational inference algorithm, and further improve its efficiency by reformulating the variational updates in the frequency domain. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method in multi-rank determination as well as its superiority in image denoising and background modeling over state-of-the-art approaches.

Low communication high performance ab initio density matrix renormalization group algorithms

Abstract: There has been recent interest in the deployment of ab initio density matrix renormalization group computations on high performance computing platforms. Here, we introduce a reformulation of the conventional distributed memory ab initio DMRG algorithm that connects it to the conceptually simpler and advantageous sum of sub-Hamiltonians approach. Starting from this framework, we further explore a hierarchy of parallelism strategies, that includes (i) parallelism over the sum of sub-Hamiltonians, (ii) parallelism over sites, (iii) parallelism over normal and complementary operators, (iv) parallelism over symmetry sectors, and (v) parallelism over dense matrix multiplications. We describe how to reduce processor load imbalance and the communication cost of the algorithm to achieve higher efficiencies. We illustrate the performance of our new open-source implementation on a recent benchmark ground-state calculation of benzene in an orbital space of 108 orbitals and 30 electrons, with a bond dimension of up to 6000, and a model of the FeMo cofactor with 76 orbitals and 113 electrons. The observed parallel scaling from 448 to 2800 CPU cores is nearly ideal.

Deep Learning-Enabled Sparse Industrial Crowdsensing and Prediction

Abstract: Mobile Crowdsensing (MCS) is a powerful sensing paradigm, which provides sufficient social data for cognitive analytics in industrial sensing, and industrial manufacturing. Considering the sensing costs, sparse MCS, as a variant, only senses the data in a few subareas, and then infers the data of unsensed subareas by the spatio-temporal relationship of the sensed data. Existing works usually assume that the sensed data are linearly spatiotemporal dependent, which cannot work well in real-world nonlinear systems, and thus, result in low data inference accuracy. Moreover, in many cases, users not only require inferring the current data, but also have an interest in predicting the near future, which can provide more information for users’ decision making. Facing these problems, we propose a deep learning-enabled industrial sensing, and prediction scheme based on sparse MCS, which consists of two parts: matrix completion and future prediction. Our goal is to achieve high-precision prediction of future moments under the hypothesis of sparse historical data. To make full use of the sparse data for prediction, we first propose a deep matrix factorization method, which can retain the nonlinear temporal-spatial relationship, and perform high-precision matrix completion. In order to predict the subareas’ data in several future sensing cycles, we further propose a nonlinear autoregressive neural network, and a stacked denoising autoencoder to obtain the temporal–spatial correlation between the data from different cycles or subareas. According to the results gained by experiments on four real-world industrial sensing datasets consisting of six typical tasks, it can be seen that the method in this article improves the accuracy of prediction using sparse data.

SpaceA: Sparse Matrix Vector Multiplication on Processing-in-Memory Accelerator

Abstract: Sparse matrix-vector multiplication (SpMV) is an important primitive across a wide range of application domains such as scientific computing and graph analytics. Due to its intrinsic memory-bound characteristics, the performance of SpMV on throughput-oriented architectures such as GPU is bounded by the limited bandwidth between processors and memory. Processing-in-memory (PIM) architectures, made feasible by advances in 3D stacking, provide new opportunities to utilize ultra-high bandwidth by integrating compute-logic into memory.In this paper, we develop an SpMV accelerator, named as SpaceA, based on PIM architectures. SpaceA integrates compute logic near memory banks to exploit bank-level bandwidth. SpaceA contains both hardware and data-mapping design features to alleviate irregular memory access patterns which hinder full utilization of high memory bandwidth. In terms of hardware design features, SpaceA consists of two unique features: (1) it utilizes the capability of outstanding memory requests to hide the memory access latency to data located in non-local memory banks; (2) it integrates Content Addressable Memory (CAM) at the bank level to exploit data reuse of the input vectors. In addition, we develop a mapping scheme that partitions the sparse matrix into different memory banks, to maximize the data locality of the input vector and to achieve workload balance among processing elements (PEs) near each bank. Overall, SpaceA together with the proposed mapping method achieves 13.54x speedup and 87.49% energy saving on average over the GPU baseline on SpMV computation. In addition to SpMV primitives, we conduct a case study on graph analytics to demonstrate the benefits of SpaceA for applications built on SpMV. Compared to Tesseract and GraphP, state-of-the-art graph accelerators, SpaceA obtains better performance due to its higher effective bandwidth provided by near-bank integration.

Deep Non-Negative Matrix Factorization Architecture Based on Underlying Basis Images Learning

Abstract: The non-negative matrix factorization (NMF) algorithm represents the original image as a linear combination of a set of basis images. This image representation method is in line with the idea of “parts constitute a whole” in human thinking. The existing deep NMF performs deep factorization on the coefficient matrix. In these methods, the basis images used to represent the original image is essentially obtained by factorizing the original images once. To extract features reflecting the deep localization characteristics of images, a novel deep NMF architecture based on underlying basis images learning is proposed for the first time. The architecture learns the underlying basis images by deep factorization on the basis images matrix. The deep factorization architecture proposed in this paper has strong interpretability. To implement this architecture, this paper proposes a deep non-negative basis matrix factorization algorithm to obtain the underlying basis images. Then, the objective function is established with an added regularization term, which directly constrains the basis images matrix to obtain the basis images with good local characteristics, and a regularized deep non-negative basis matrix factorization algorithm is proposed. The regularized deep nonlinear non-negative basis matrix factorization algorithm is also proposed to handle pattern recognition tasks with complex data. This paper also theoretically proves the convergence of the algorithm. Finally, the experimental results show that the deep NMF architecture based on the underlying basis images learning proposed in this paper can obtain better recognition performance than the other state-of-the-art methods.

A Hyperspectral Anomaly Detection Method Based on Low-Rank and Sparse Decomposition With Density Peak Guided Collaborative Representation

Abstract: The low-rank and sparse decomposition model (LSDM) has been widely studied by researchers and has successfully solved the problem of hyperspectral image (HSI) anomaly detection (AD) The traditional LSDM usually ignores the information of the low-rank matrix, which only detects the anomalous targets by using the sparse component To utilize both the sparse component and the low-rank component comprehensively, an anomaly detector for HSIs based on LSDM with density peak guided collaborative representation (LSDDPCRD) is proposed in this article First, the LSDM technique with the mixture of Gaussian model is used to decompose the original HSI, which can also alleviate the background noise contamination problem Then, the low-rank matrix is detected by the density peak guided collaborative representation detection algorithm, while the sparse matrix is calculated according to the Manhattan distance In addition, an entropy-based adaptive fusing method is designed to combine the results obtained from the low-rank matrix and the sparse component It could choose the fusing weights adaptively according to the characteristics of an HSI The experimental results indicate that the LSDDPCRD performs better than eight classical and state-of-the-art AD algorithms (GRX, LRX, SRX-Segmented, CRD, RPCA-RX, LSMAD, LRASR, and LSDM-MoG) on four real HSIs

Accommodating Transformer onto FPGA: Coupling the Balanced Model Compression and FPGA-Implementation Optimization

Abstract: Recently, Transformers gradually gain popularity and perform outstanding for many Natural Language Processing (NLP) tasks. However, Transformers suffer from heavy computation and memory footprint, making it difficult to deploy on embedded devices. The field-programmable gate array (FPGA) is widely used to accelerate deep learning algorithms for its advantages. However, the trained Transformer models are too large to accommodate to an FPGA fabric. To accommodate Transformer onto FPGA and achieve efficient execution, we propose an acceleration framework coupling the balanced model compression at the algorithm level and FPGA-implementation optimization at the hardware level. At algorithm level, we adopt a block-balanced pruning and propose an efficient sparse matrix storage format for this pruning technique, named Compressed Block Row (CBR). At the hardware level, we design an accelerator for sparse model. And we also abstract a performance analytic model to evaluate the performance of accelerator. Experiments show that our CBR format perform better than general formats and can significantly save storage space. And our accelerator can achieve $38\times$ and $1.93\times$ speedup compared to other works on CPU and GPU respectively.