scispace - formally typeset
Search or ask a question

Showing papers in "IEEE Transactions on Signal Processing in 2019"


Journal ArticleDOI
TL;DR: A new spectral domain convolutional architecture for deep learning on graphs with a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest.
Abstract: The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains. In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs. The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest. Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely connected graphs, and can handle different constructions of Laplacian operators. Extensive experimental results show the superior performance of our approach, in comparison to other spectral domain convolutional architectures, on spectral image classification, community detection, vertex classification, and matrix completion tasks.

512 citations


Journal ArticleDOI
TL;DR: Li et al. as mentioned in this paper proposed two different deep architectures: a standard fully connected multi-layer network, and a detection network (DetNet), which was specifically designed for the task of multiple-input-multiple-output detection.
Abstract: In this paper, we consider multiple-input-multiple-output detection using deep neural networks. We introduce two different deep architectures: a standard fully connected multi-layer network, and a detection network (DetNet), which is specifically designed for the task. The structure of DetNet is obtained by unfolding the iterations of a projected gradient descent algorithm into a network. We compare the accuracy and runtime complexity of the proposed approaches and achieve state-of-the-art performance while maintaining low computational requirements. Furthermore, we manage to train a single network to detect over an entire distribution of channels. Finally, we consider detection with soft outputs and show that the networks can easily be modified to produce soft decisions.

412 citations


Journal ArticleDOI
TL;DR: This tutorial-style overview highlights the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees and reviews two contrasting approaches: two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and global landscape analysis and initialization-free algorithms.
Abstract: Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative methods such as gradient descent have been remarkably successful in practice. The theoretical footings, however, had been largely lacking until recently. In this tutorial-style overview, we highlight the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees. We review two contrasting approaches: (1) two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and (2) global landscape analysis and initialization-free algorithms. Several canonical matrix factorization problems are discussed, including but not limited to matrix sensing, phase retrieval, matrix completion, blind deconvolution, and robust principal component analysis. Special care is taken to illustrate the key technical insights underlying their analyses. This article serves as a testament that the integrated consideration of optimization and statistics leads to fruitful research findings.

369 citations


Journal ArticleDOI
TL;DR: Two architectures that generalize convolutional neural networks (CNNs) for the processing of signals supported on graphs are introduced and Multinode aggregation GNNs are consistently the best-performing GNN architecture for operation in large-scale graphs.
Abstract: Two architectures that generalize convolutional neural networks (CNNs) for the processing of signals supported on graphs are introduced. We start with the selection graph neural network (GNN), which replaces linear time invariant filters with linear shift invariant graph filters to generate convolutional features and reinterprets pooling as a possibly nonlinear subsampling stage where nearby nodes pool their information in a set of preselected sample nodes. A key component of the architecture is to remember the position of sampled nodes to permit computation of convolutional features at deeper layers. The second architecture, dubbed aggregation GNN, diffuses the signal through the graph and stores the sequence of diffused components observed by a designated node. This procedure effectively aggregates all components into a stream of information having temporal structure to which the convolution and pooling stages of regular CNNs can be applied. A multinode version of aggregation GNNs is further introduced for operation in large-scale graphs. An important property of selection and aggregation GNNs is that they reduce to conventional CNNs when particularized to time signals reinterpreted as graph signals in a circulant graph. Comparative numerical analyses are performed in a source localization application over synthetic and real-world networks. Performance is also evaluated for an authorship attribution problem and text category classification. Multinode aggregation GNNs are consistently the best-performing GNN architecture.

291 citations


Journal ArticleDOI
TL;DR: A 3D-structured orthogonal matching pursuit algorithm based channel estimation technique to solve the downlink channel estimation problem for OTFS massive MIMO.
Abstract: Orthogonal time frequency space (OTFS) modulation outperforms orthogonal frequency division multiplexing (OFDM) in high-mobility scenarios. One challenge for OTFS massive MIMO is downlink channel estimation due to the large number of base station antennas. In this paper, we propose a 3D-structured orthogonal matching pursuit algorithm based channel estimation technique to solve this problem. First, we show that the OTFS MIMO channel exhibits 3D-structured sparsity: normal sparsity along the delay dimension, block sparsity along the Doppler dimension, and burst sparsity along the angle dimension. Based on the 3D-structured channel sparsity, we then formulate the downlink channel estimation problem as a sparse signal recovery problem. Simulation results show that the proposed algorithm can achieve accurate channel state information with low pilot overhead.

223 citations


Journal ArticleDOI
TL;DR: DNNs are trained here with a model-free primal-dual method that simultaneously learns a DNN parameterization of the resource allocation policy and optimizes the primal and dual variables.
Abstract: This paper considers the design of optimal resource allocation policies in wireless communication systems, which are generically modeled as a functional optimization problem with stochastic constraints. These optimization problems have the structure of a learning problem in which the statistical loss appears as a constraint, motivating the development of learning methodologies to attempt their solution. To handle stochastic constraints, training is undertaken in the dual domain. It is shown that this can be done with small loss of optimality when using near-universal learning parameterizations. In particular, since deep neural networks (DNNs) are near universal, their use is advocated and explored. DNNs are trained here with a model-free primal-dual method that simultaneously learns a DNN parameterization of the resource allocation policy and optimizes the primal and dual variables. Numerical simulations demonstrate the strong performance of the proposed approach on a number of common wireless resource allocation problems.

223 citations


Journal ArticleDOI
TL;DR: This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and nonsmooth terms that is as good as one of the two convergence rates that match the typical rates for the general gradient descent and the consensus averaging.
Abstract: This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and nonsmooth terms. Specifically, the smooth and nonsmooth terms are dealt with by gradient and proximal updates, respectively. The proposed algorithm is closely related to a previous algorithm, PG-EXTRA (W. Shi, Q. Ling, G. Wu, and W. Yin, “A proximal gradient algorithm for decentralized composite optimization,” IEEE Trans. Signal Process., vol. 63, no. 22, pp. 6013-6023, 2015), but has a few advantages. First of all, agents use uncoordinated step-sizes, and the stable upper bounds on step-sizes are independent of network topologies. The step-sizes depend on local objective functions, and they can be as large as those of the gradient descent. Second, for the special case without nonsmooth terms, linear convergence can be achieved under the strong convexity assumption. The dependence of the convergence rate on the objective functions and the network are separated, and the convergence rate of the new algorithm is as good as one of the two convergence rates that match the typical rates for the general gradient descent and the consensus averaging. We provide numerical experiments to demonstrate the efficacy of the introduced algorithm and validate our theoretical discoveries.

221 citations


Journal ArticleDOI
TL;DR: A new sparse array configuration based on the maximum inter-element spacing constraint (MISC) is proposed, which enjoys two important advantages, namely, providing a higher number of DOFs and reducing the mutual coupling effects.
Abstract: Recently, nested and coprime arrays have attracted considerable interest due to their capability of providing increased array aperture, enhanced degrees of freedom (DOFs), and reduced mutual coupling effect compared to uniform linear arrays (ULAs). These features are critical to improving the performance of direction-of-arrival estimation and adaptive beamforming. In this paper, a new sparse array configuration based on the maximum inter-element spacing constraint (MISC) is proposed. The MISC array configuration generally consists of three sparse ULAs plus two separate sensors that are appropriately placed. The MISC array configurations are designed based on the inter-element spacing set, which, for a given number of sensors, is uniquely determined by a closed-form expression. We also derive closed-form expressions for the number of uniform DOFs of the MISC arrays with any number of sensors. Compared with the existing sparse arrays, the MISC array enjoys two important advantages, namely, providing a higher number of DOFs and reducing the mutual coupling effects. Numerical simulations are conducted to verify the superiority of the MISC array over other sparse arrays.

185 citations


Journal ArticleDOI
TL;DR: Ntakaris et al. as discussed by the authors developed a large-scale deep learning model to predict price movements from limit order book (LOB) data of cash equities, which utilizes convolutional filters to capture the spatial structure of the LOBs as well as long short-term memory modules to capture longer time dependencies.
Abstract: We develop a large-scale deep learning model to predict price movements from limit order book (LOB) data of cash equities. The architecture utilizes convolutional filters to capture the spatial structure of the LOBs as well as long short-term memory modules to capture longer time dependencies. The proposed network outperforms all existing state-of-the-art algorithms on the benchmark LOB dataset [A. Ntakaris, M. Magris, J. Kanniainen, M. Gabbouj, and A. Iosifidis, “Benchmark dataset for mid-price prediction of limit order book data with machine learning methods,” J. Forecasting , vol. 37, no. 8, 852–866, 2018]. In a more realistic setting, we test our model by using one-year market quotes from the London Stock Exchange, and the model delivers a remarkably stable out-of-sample prediction accuracy for a variety of instruments. Importantly, our model translates well to instruments that were not part of the training set, indicating the model's ability to extract universal features. In order to better understand these features and to go beyond a “black box” model, we perform a sensitivity analysis to understand the rationale behind the model predictions and reveal the components of LOBs that are most relevant. The ability to extract robust features that translate well to other instruments is an important property of our model, which has many other applications.

176 citations


Journal ArticleDOI
TL;DR: In this article, a generic extension of variational mode decomposition (VMD) algorithm to multivariate or multichannel data is presented, which utilizes a model for multivariate modulated oscillations that is based on the presence of a joint or common frequency component among all channels of input data.
Abstract: We present a generic extension of variational mode decomposition (VMD) algorithm to multivariate or multichannel data. The proposed method utilizes a model for multivariate modulated oscillations that is based on the presence of a joint or common frequency component among all channels of input data. We then formulate a variational optimization problem that aims to extract an ensemble of band-limited modes containing inherent multivariate modulated oscillations present in the data. The cost function to be minimized is the sum of bandwidths of all signal modes across all input data channels, which is a generic extension of the cost function used in standard VMD to multivariate data. Minimization of the resulting variational model is achieved through the alternating direction method of multipliers (ADMM) that yields an optimal set of multivariate modes in terms of narrow bandwidth and corresponding center frequencies. The proposed extension is elegant as it does not require any extra user-defined parameters for its operation i.e., it uses the same parameters as standard VMD. We demonstrate the effectiveness of the proposed method through results obtained from extensive simulations involving test (synthetic) and real world multivariate data sets. Specifically, we highlight the utility of the proposed method in two real world applications which include the separation of alpha rhythms in multivariate electroencephalogram (EEG) data and the decomposition of bivariate cardiotocographic signals that consist of fetal heart rate and maternal uterine contraction (FHR-UC) as its two channels.

169 citations


Journal ArticleDOI
TL;DR: A novel model-specific DNN is developed for real-time PSSE requiring only offline training and minimal tuning effort and outperforms nearly by an order-of-magnitude its competing alternatives, including the widely adopted Gauss–Newton PSSE solver.
Abstract: Contemporary power grids are being challenged by rapid and sizeable voltage fluctuations that are caused by large-scale deployment of renewable generators, electric vehicles, and demand response programs. In this context, monitoring the grid's operating conditions in real time becomes increasingly critical. With the emergent large scale and nonconvexity, existing power system state estimation (PSSE) schemes become computationally expensive or often yield suboptimal performance. To bypass these hurdles, this paper advocates physics-inspired deep neural networks (DNNs) for real-time power system monitoring. By unrolling an iterative solver that was originally developed using the exact ac model, a novel model-specific DNN is developed for real-time PSSE requiring only offline training and minimal tuning effort. To further enable system awareness, even ahead of the time horizon, as well as to endow the DNN-based estimator with resilience, deep recurrent neural networks (RNNs) are also pursued for power system state forecasting. Deep RNNs leverage the long-term nonlinear dependencies present in the historical voltage time series to enable forecasting, and they are easy to implement. Numerical tests showcase improved performance of the proposed DNN-based estimation and forecasting approaches compared with existing alternatives. In real load data experiments on the IEEE 118-bus benchmark system, the novel model-specific DNN-based PSSE scheme outperforms nearly by an order-of-magnitude its competing alternatives, including the widely adopted Gauss–Newton PSSE solver.

Journal ArticleDOI
TL;DR: The novel GSTM distributed Kalman filter has the important advantage over the RSTKF that the adaptation of the mixing parameter is much more straightforward than learning the degrees of freedom parameter.
Abstract: In this paper, a novel Gaussian–Student's t mixture (GSTM) distribution is proposed to model non-stationary heavy-tailed noises. The proposed GSTM distribution can be formulated as a hierarchical Gaussian form by introducing a Bernoulli random variable, based on which a new hierarchical linear Gaussian state-space model is constructed. A novel robust GSTM distribution based Kalman filter is proposed based on the constructed hierarchical linear Gaussian state-space model using the variational Bayesian approach. The Kalman filter and robust Student's t based Kalman filter (RSTKF) with fixed distribution parameters are two existing special cases of the proposed filter. The novel GSTM distributed Kalman filter has the important advantage over the RSTKF that the adaptation of the mixing parameter is much more straightforward than learning the degrees of freedom parameter. Simulation results illustrate that the proposed filter has better estimation accuracy than those of the Kalman filter and RSTKF for a linear state-space model with non-stationary heavy-tailed noises.

Journal ArticleDOI
TL;DR: The proposed iterative channel estimation algorithm based on the least square estimation (LSE) and sparse message passing (SMP) algorithm for the millimeter wave (mmWave) MIMO systems has much better performance than the existing sparse estimators, especially when the channel is sparse.
Abstract: We propose an iterative channel estimation algorithm based on the least square estimation (LSE) and sparse message passing (SMP) algorithm for the millimeter wave (mmWave) MIMO systems. The channel coefficients of the mmWave MIMO are approximately modeled as a Bernoulli–Gaussian distribution and the channel matrix is sparse with only a few nonzero entries. By leveraging the advantage of sparseness, we propose an algorithm that iteratively detects the exact locations and values of nonzero entries of the sparse channel matrix. At each iteration, the locations are detected by the SMP, and values are estimated with the LSE. We also analyze the Cramer–Rao Lower Bound (CLRB), and show that the proposed algorithm is a minimum variance unbiased estimator under the assumption that we have the partial priori knowledge of the channel. Furthermore, we employ the Gaussian approximation for message densities under density evolution to simplify the analysis of the algorithm, which provides a simple method to predict the performance of the proposed algorithm. Numerical experiments show that the proposed algorithm has much better performance than the existing sparse estimators, especially when the channel is sparse. In addition, our proposed algorithm converges to the CRLB of the genie-aided estimation of sparse channels with only five turbo iterations.

Journal ArticleDOI
TL;DR: This work addresses the waveform design problem for multiple-input multiple-output (MIMO) radar in spectrally crowded environments and proposes an algorithm based on first-order Taylor series approximation as well as minorization–maximization (MM) based algorithms to design theSpectrally constrained waveforms.
Abstract: We address the waveform design problem for multiple-input multiple-output (MIMO) radar in spectrally crowded environments. We exploit the mutual information between the target reflections and the target responses as the design metric. To tackle the associated nonconvex optimization problem, we propose an algorithm based on first-order Taylor series approximation as well as minorization–maximization (MM) based algorithms to design the spectrally constrained waveforms. Interestingly, for some scenarios, we can synthesize the globally optimal (spectrally constrained) waveforms with the maximum mutual information. We also show that, through intelligent waveform design, MIMO radar can coexist more efficiently with other communication systems occupying the same spectrum, while suffering from insignificant mutual information losses.

Journal ArticleDOI
TL;DR: This paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general.
Abstract: Neural networks with rectified linear unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being, e.g., Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory.

Journal ArticleDOI
TL;DR: In this paper, the authors proposed a trainable iterative soft thresholding algorithm (TISTA), which consists of two estimation units: a linear estimation unit and a minimum mean squared error estimator based shrinkage unit.
Abstract: In this paper, we propose a novel sparse signal recovery algorithm called the trainable iterative soft thresholding algorithm (TISTA). The proposed algorithm consists of two estimation units: a linear estimation unit and a minimum mean squared error (MMSE) estimator based shrinkage unit. The error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables that control step size and the error variance for the MMSE shrinkage function. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is nearly equal to the number of iteration rounds and is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices, such as Gaussian matrices, binary matrices, and matrices with large condition numbers. Numerical results also demonstrate that, in many cases, TISTA provides significantly faster convergence than approximate message passing (AMP) and the learned iterative shrinkage thresholding algorithm and also outperforms orthogonal AMP in the NMSE performance.

Journal ArticleDOI
TL;DR: In this paper, a sparse Bayesian learning (SBL)-based method for estimating the direction of arrival (DOA) in a multiple-input and multiple-output (MIMO) radar system with unknown mutual coupling effect between antennas is investigated.
Abstract: In the practical radar with multiple antennas, the antenna imperfections degrade the system performance. In this paper, the problem of estimating the direction of arrival (DOA) in a multiple-input and multiple-output (MIMO) radar system with unknown mutual coupling effect between antennas is investigated. To exploit the target sparsity in the spatial domain, the compressed sensing based methods have been proposed by discretizing the detection area and formulating the dictionary matrix, so an off-grid gap is caused by the discretization processes. In this paper, different from the present DOA estimation methods, both the off-grid gap due to the sparse sampling and the unknown mutual coupling effect between antennas are considered at the same time, and a novel sparse system model for DOA estimation is formulated. Then, a novel sparse Bayesian learning (SBL)-based method named sparse Bayesian learning with the mutual coupling (SBLMC) is proposed, where an expectation-maximum-based method is established to estimate all the unknown parameters including the noise variance, the mutual coupling vectors, the off-grid vector, and the variance vector of scattering coefficients. Additionally, the prior distributions for all the unknown parameters are theoretically derived. With regard to the DOA estimation performance, the proposed SBLMC method can outperform state-of-the-art methods in the MIMO radar with unknown mutual coupling effect, while keeping the acceptable computational complexity.

Journal ArticleDOI
TL;DR: This paper considers covert communications in the context of unmanned aerial vehicle (UAV) networks, aiming to hide a UAV for transmitting critical information out of an area that is monitored and where communication is not allowed, and develops a joint trajectory and transmit power optimization scheme.
Abstract: This paper considers covert communications in the context of unmanned aerial vehicle (UAV) networks, aiming to hide a UAV for transmitting critical information out of an area that is monitored and where communication is not allowed. Specifically, the UAV as a transmitter intends to transmit information to a legitimate receiver (Bob) covertly while avoiding being detected by a warden (Willie), where location uncertainty exists at Bob and/or Willie. In order to enhance the considered covert communication performance, we jointly optimize the UAV's trajectory and transmit power in terms of maximizing the average covert transmission rate from the UAV to Bob subject to transmission outage constraint and covertness constraint. The formulated optimization problem is difficult to tackle directly due to the intractable constraints. As such, we first employ conservative approximation to transform a constraint into a deterministic form and then apply the first-order restrictive approximation to transform the optimization problem into a convex form. By applying the successive convex approximation technique, an efficient iterative algorithm is developed to solve the optimization problem. Our examination shows that the developed joint trajectory and transmit power optimization scheme achieves significantly better covert communication performance as compared to a benchmark scheme.

Journal ArticleDOI
TL;DR: In this paper, a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies was developed, which is applicable to locally balanced combination matrices which are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties.
Abstract: This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range and superior convergence performance than the EXTRA strategy. The exact diffusion method is applicable to locally balanced left-stochastic combination matrices which, compared to the conventional doubly stochastic matrix, are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties. The derivation of the exact diffusion strategy relies on reformulating the aggregate optimization problem as a penalized problem and resorting to a diagonally weighted incremental construction. Detailed stability and convergence analyses are pursued in Part II of this paper and are facilitated by examining the evolution of the error dynamics in a transformed domain. Numerical simulations illustrate the theoretical conclusions.

Journal ArticleDOI
TL;DR: Sparse array motion is utilized to increase the numbers of achievable both degrees of freedom (DOFs) and consecutive lags in direction-of-arrival (DOA) estimation problems and shows the respective DOA estimation performance based on sparse reconstruction techniques.
Abstract: This paper utilizes sparse array motion to increase the numbers of achievable both degrees of freedom (DOFs) and consecutive lags in direction-of-arrival (DOA) estimation problems. We use commonly employed environment-independent sparse array configurations. The design of these arrays is not dependent on the sources in the field of view, but rather aims at achieving desirable difference co-arrays. They include structured coprime and nested arrays, minimum redundancy array (MRA), minimum hole array (MHA), and sparse uniform linear array (SULA). Array motion can fill the holes in the spatial autocorrelation lags associated with a fixed platform and, therefore, increases the number of sources detectable by the same number of array sensors. Quasi-stationarity of the environment is assumed where the source locations and waveforms are considered invariant over array motion of half wavelength. Closed-form expressions of the number of DOFs and consecutive spatial correlation lags for coprime and nested arrays as well as SULA, due to array translation motion, are derived. The number of DOFs and consecutive lags for the specific cases of MRA an 5 avaluated. We show the respective DOA estimation performance based on sparse reconstruction techniques.

Journal ArticleDOI
TL;DR: This paper generalizes state-of-the-art distributed graph filters to filters where every node weights the signal of its neighbors with different values while keeping the aggregation operation linear, and characterize a subset of shift-invariant graph filters that can be described with edge-variant recursions.
Abstract: Graph filters are one of the core tools in graph signal processing. A central aspect of them is their direct distributed implementation. However, the filtering performance is often traded with distributed communication and computational savings. To improve this tradeoff, this paper generalizes state-of-the-art distributed graph filters to filters where every node weights the signal of its neighbors with different values while keeping the aggregation operation linear. This new implementation, labeled as edge-variant graph filter, yields a significant reduction in terms of communication rounds while preserving the approximation accuracy. In addition, we characterize a subset of shift-invariant graph filters that can be described with edge-variant recursions. By using a low-dimensional parameterization, these shift-invariant filters provide new insights in approximating linear graph spectral operators through the succession and composition of local operators, i.e., fixed support matrices. A set of numerical results shows the benefits of the edge-variant graph filters over current methods and illustrates their potential to a wider range of applications than graph filtering.

Journal ArticleDOI
TL;DR: Th thinned coprime array (TCA) is presented, which exploits the redundancy in the structure of existing coprimes array and achieves the same virtual aperture and degrees of freedom as the conventional coprIME array with much fewer number of sensors.
Abstract: In this work, we present a new coprime array structure termed thinned coprime array (TCA), which exploits the redundancy in the structure of existing coprime array and achieves the same virtual aperture and degrees of freedom (DOFs) as the conventional coprime array with much fewer number of sensors. In comparison to other sparse arrays, thinned coprime arrays possess more unique lags (total number of difference co-arrays) than the nested arrays, while the number of consecutive lags (connected co-arrays) generated is close to 75% of the consecutive lags of the nested arrays with hole-free co-arrays. The resulting structure is much sparser and the number of sensor pairs with small separation is significantly reduced. Theoretical properties and proofs are provided and simulations are presented to demonstrate its robustness against heavy levels of mutual coupling using compressive sensing based direction of arrival estimation as well as certain additional desirable characteristics.

Journal ArticleDOI
TL;DR: A channel estimation scheme for frequency-division duplex (FDD) mmWave massive MIMO-OFDM systems with hybrid analog/digital precoding, which takes the beam squint effect into consideration is proposed and numerical results demonstrate the superiority of the proposed scheme over the conventional methods under general system configurations in mmWave communications.
Abstract: With the increasing scale of antenna arrays in wideband millimeter-wave (mmWave) communications, the physical propagation delays of electromagnetic waves traveling across the whole array will become large and comparable to the time-domain sample period, which is known as the spatial-wideband effect. In this case, different subcarriers in an orthogonal frequency division multiplexing (OFDM) system will “see” distinct angles of arrival (AoAs) for the same path. This effect is known as beam squint , resulting from the spatial-wideband effect, and makes the approaches based on the conventional multiple-input multiple-output (MIMO) model, such as channel estimation and precoding, inapplicable. After discussing the relationship between beam squint and the spatial-wideband effect, we propose a channel estimation scheme for frequency-division duplex (FDD) mmWave massive MIMO-OFDM systems with hybrid analog/digital precoding, which takes the beam squint effect into consideration. A compressive sensing-based approach is developed to extract the frequency-insensitive parameters of each uplink channel path, i.e., the AoA and the time delay, and the frequency-sensitive parameter, i.e., the complex channel gain. With the help of the reciprocity of these frequency-insensitive parameters in FDD systems, the downlink channel estimation can be greatly simplified, where only limited pilots are needed to obtain downlink complex gains and reconstruct downlink channels. Furthermore, the uplink and downlink channel covariance matrices can be constructed from these frequency-insensitive channel parameters rather than through a long-term average, which enables the minimum mean-squared error (MMSE) channel estimation to further enhance performance. Numerical results demonstrate the superiority of the proposed scheme over the conventional methods under general system configurations in mmWave communications.

Journal ArticleDOI
TL;DR: The proximal online gradient descent (OGD) algorithm for tracking the optimum of a composite objective function comprising of a differentiable loss function and a nondifferentiable regularizer and is generalized for application to large-scale problems where the loss function has a finite sum structure.
Abstract: We consider nondifferentiable dynamic optimization problems such as those arising in robotics and subspace tracking. Given the computational constraints and the time-varying nature of the problem, a low-complexity algorithm is desirable, while the accuracy of the solution may only increase slowly over time. We put forth the proximal online gradient descent (OGD) algorithm for tracking the optimum of a composite objective function comprising of a differentiable loss function and a nondifferentiable regularizer. An online learning framework is considered and the gradient of the loss function is allowed to be erroneous. Both, the gradient error as well as the dynamics of the function optimum or target are adversarial and the performance of the inexact proximal OGD is characterized in terms of its dynamic regret, expressed in terms of the cumulative error and path length of the target. The proposed inexact proximal OGD is generalized for application to large-scale problems where the loss function has a finite sum structure. In such cases, evaluation of the full gradient may not be viable and a variance reduced version is proposed that allows the component functions to be subsampled. The efficacy of the proposed algorithms is tested on the problem of formation control in robotics and on the dynamic foreground–background separation problem in video.

Journal ArticleDOI
TL;DR: A framework for approximating the optimal fully digital precoder with a feasible hybrid one, and a special case of Alt-MaG, minimal gap iterative quantization (MaGiQ), that results in low complexity and lower mean squared error (MSE) than other common methods, in the case of very few RF chains.
Abstract: In massive MIMO systems, hybrid beamforming is an essential technique for exploiting the potential array gain without using a dedicated RF chain for each antenna. In this paper, we consider the data phase in massive MIMO communication, where the transmitter and receiver use fewer RF chains than antennas. We examine several different fully and partially connected schemes and consider the design of hybrid beamformers that minimize the estimation error in the data. For the hybrid precoder, we introduce a framework for approximating the optimal fully digital precoder with a feasible hybrid one. We exploit the fact that the fully digital precoder is unique only up to a unitary matrix and optimize over this matrix and the hybrid precoder alternately. Our alternating minimization of approximation gap (Alt-MaG) framework improves the performance over state-of-the-art methods with no substantial increase in complexity. In addition, we present a special case of Alt-MaG, minimal gap iterative quantization (MaGiQ), that results in low complexity and lower mean squared error (MSE) than other common methods, in the case of very few RF chains. MaGiQ is also shown to coincide with the optimal fully digital solution in some scenarios. For combiner design, we exploit the structure of the MSE objective and develop a greedy ratio trace maximization technique that achieves low MSE under various settings. All of our algorithms can be used with multiple hardware architectures.

Journal ArticleDOI
TL;DR: The Quantized Decentralized Gradient Descent algorithm is proposed, in which nodes update their local decision variables by combining the quantized information received from their neighbors with their local information, and it is proved that under standard strong convexity and smoothness assumptions for the objective function, QDGD achieves a vanishing mean solution error under customary conditions for quantizers.
Abstract: We consider the problem of decentralized consensus optimization, where the sum of $n$ smooth and strongly convex functions are minimized over $n$ distributed agents that form a connected network. In particular, we consider the case that the communicated local decision variables among nodes are quantized in order to alleviate the communication bottleneck in distributed optimization. We propose the Quantized Decentralized Gradient Descent (QDGD) algorithm, in which nodes update their local decision variables by combining the quantized information received from their neighbors with their local information. We prove that under standard strong convexity and smoothness assumptions for the objective function, QDGD achieves a vanishing mean solution error under customary conditions for quantizers. To the best of our knowledge, this is the first algorithm that achieves vanishing consensus error in the presence of quantization noise. Moreover, we provide simulation results that show tight agreement between our derived theoretical convergence rate and the numerical results.

Journal ArticleDOI
TL;DR: This paper first proves that each path component of the wideband beamspace channel exhibits a unique frequency-dependent sparse structure, and then proposes a successive support detection (SSD) based beam space channel estimation scheme, which successively estimates all the sparse path components following the classical idea of successive interference cancellation.
Abstract: Beamspace channel estimation is indispensable for millimeter-wave MIMO systems relying on lens antenna arrays for achieving substantially increased data rates, despite using a small number of radio-frequency chains. However, most of the existing beamspace channel estimation schemes have been designed for narrowband systems, while the rather scarce wideband solutions tend to assume that the sparse beamspace channel exhibits a common support in the frequency domain, which has a limited validity owing to the effect of beam squint caused by the wide bandwidth in practice. In this paper, we investigate the wideband beamspace channel estimation problem without the common support assumption. Specifically, by exploiting the effect of beam squint, we first prove that each path component of the wideband beamspace channel exhibits a unique frequency-dependent sparse structure. Inspired by this structure, we then propose a successive support detection (SSD) based beamspace channel estimation scheme, which successively estimates all the sparse path components following the classical idea of successive interference cancellation. For each path component, its support at different frequencies is jointly estimated to improve the accuracy by utilizing the proved sparse structure, and its influence is removed to estimate the remaining path components. The performance analysis shows that the proposed SSD-based scheme can accurately estimate the wideband beamspace channel at a low complexity. Simulation results verify that the proposed SSD-based scheme enjoys a reduced pilot overhead, and yet achieves an improved channel estimation accuracy.

Journal ArticleDOI
TL;DR: This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time using the emerging time-varying convex optimization formalism.
Abstract: This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging time-varying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constraints. Departing from existing batch and feed-forward optimization approaches, the design of the algorithms capitalizes on an online implementation of primal-dual projected-gradient methods; the gradient steps are, however, suitably modified to accommodate feedback from the system in the form of measurements, hence, the term “online optimization with feedback.” By virtue of this approach, the resultant algorithms can cope with model mismatches in the algebraic representation of the system states and outputs, they avoid pervasive measurements of exogenous inputs, and they naturally lend themselves to a distributed implementation. Under suitable assumptions, analytical convergence claims are established in terms of dynamic regret. Furthermore, when the synthesis of the feedback-based online algorithms is based on a regularized Lagrangian function, $\boldsymbol{Q}$ -linear convergence to solutions of the time-varying optimization problem is shown.

Journal ArticleDOI
TL;DR: In this paper, a low-complexity iterative linear minimum mean square error (LMMSE) multiuser detector for the multiple-input and multiple-output system with nonorthogonal multiple access (MIMO-NOMA) was proposed.
Abstract: This paper considers a low-complexity iterative linear minimum mean square error (LMMSE) multiuser detector for the multiple-input and multiple-output system with nonorthogonal multiple access (MIMO-NOMA), where multiple single-antenna users simultaneously communicate with a multiple-antenna base station (BS). While LMMSE being a linear detector has a low complexity, it has suboptimal performance in multiuser detection scenario due to the mismatch between LMMSE detection and multiuser decoding. Therefore, in this paper, we provide the matching conditions between the detector and decoders for MIMO-NOMA, which are then used to derive the achievable rate of the iterative detection. We prove that a matched iterative LMMSE detector can achieve the optimal capacity of symmetric MIMO-NOMA with any number of users, the optimal sum capacity of asymmetric MIMO-NOMA with any number of users, all the maximal extreme points in the capacity region of asymmetric MIMO-NOMA with any number of users, and all points in the capacity region of two-user and three-user asymmetric MIMO-NOMA systems. In addition, a kind of practical low-complexity error-correcting multiuser code, called irregular repeat-accumulate code, is designed to match the LMMSE detector. Numerical results shows that the bit error rate performance of the proposed iterative LMMSE detection outperforms the state-of-art methods and is within 0.8 dB from the associated capacity limit.

Journal ArticleDOI
TL;DR: A new projection, descent, and retraction (PDR) update strategy is derived that allows for monotonic cost function improvement while maintaining feasibility over the complex circle manifold (constant modulus set).
Abstract: The ability of multiple-input multiple-output (MIMO) radar systems to adapt waveforms across antennas allows flexibility in the transmit beampattern design. In cognitive radar, a popular cost function is to minimize the deviation against an idealized beampattern (which is arrived at with knowledge of the environment). The optimization of the transmit beampattern becomes particularly challenging in the presence of practical constraints on the transmit waveform. One of the hardest of such constraints is the non-convex constant modulus constraint, which has been the subject of much recent work. In a departure from most existing approaches, we develop a solution that involves direct optimization over the non-convex complex circle manifold. That is, we derive a new projection, descent, and retraction (PDR) update strategy that allows for monotonic cost function improvement while maintaining feasibility over the complex circle manifold (constant modulus set). For quadratic cost functions (as is the case with beampattern deviation), we provide analytical guarantees of monotonic cost function improvement along with proof of convergence to a local minima. We evaluate the proposed PDR algorithm against other candidate MIMO beampattern design methods and show that PDR can outperform competing wideband beampattern design methods while being computationally less expensive. Finally, orthogonality across antennas is incorporated in the PDR framework by adding a penalty term to the beampattern cost function. Enabled by orthogonal waveforms, robustness to target direction mismatch is also demonstrated.