Showing papers in "IEEE Transactions on Signal Processing in 2020"

Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air

Abstract: We study federated machine learning (ML) at the wireless edge, where power- and bandwidth-limited wireless devices with local datasets carry out distributed stochastic gradient descent (DSGD) with the help of a parameter server (PS). Standard approaches assume separate computation and communication, where local gradient estimates are compressed and transmitted to the PS over orthogonal links. Following this digital approach, we introduce D-DSGD, in which the wireless devices employ gradient quantization and error accumulation, and transmit their gradient estimates to the PS over a multiple access channel (MAC). We then introduce a novel analog scheme, called A-DSGD, which exploits the additive nature of the wireless MAC for over-the-air gradient computation, and provide convergence analysis for this approach. In A-DSGD, the devices first sparsify their gradient estimates, and then project them to a lower dimensional space imposed by the available channel bandwidth. These projections are sent directly over the MAC without employing any digital code. Numerical results show that A-DSGD converges faster than D-DSGD thanks to its more efficient use of the limited bandwidth and the natural alignment of the gradient estimates over the channel. The improvement is particularly compelling at low power and low bandwidth regimes. We also illustrate for a classification problem that, A-DSGD is more robust to bias in data distribution across devices, while D-DSGD significantly outperforms other digital schemes in the literature. We also observe that both D-DSGD and A-DSGD perform better with the number of devices, showing their ability in harnessing the computation power of edge devices.

A Framework of Robust Transmission Design for IRS-Aided MISO Communications With Imperfect Cascaded Channels

Abstract: Intelligent reflection surface (IRS) has recently been recognized as a promising technique to enhance the performance of wireless systems due to its ability of reconfiguring the signal propagation environment. However, the perfect channel state information (CSI) is challenging to obtain at the base station (BS) due to the lack of radio frequency (RF) chains at the IRS. Since most of the existing channel estimation methods were developed to acquire the cascaded BS-IRS-user channels, this paper is the first work to study the robust beamforming based on the imperfect cascaded BS-IRS-user channels at the transmitter (CBIUT). Specifically, the transmit power minimization problems are formulated subject to the worst-case rate constraints under the bounded CSI error model, and the rate outage probability constraints under the statistical CSI error model, respectively. After approximating the worst-case rate constraints by using the S-procedure and the rate outage probability constraints by using the Bernstein-type inequality, the reformulated problems can be efficiently solved. Numerical results show that the negative impact of the CBIUT error on the system performance is greater than that of the direct CSI error.

Joint Transmit Beamforming for Multiuser MIMO Communications and MIMO Radar

Abstract: Future wireless communication systems are expected to explore spectral bands typically used by radar systems, in order to overcome spectrum congestion of traditional communication bands. Since in many applications radar and communication share the same platform, spectrum sharing can be facilitated by joint design as a dual-function radar-communications system. In this paper, we propose a joint transmit beamforming model for a dual-function multiple-input-multiple-output (MIMO) radar and multiuser MIMO communication transmitter. The proposed dual-function system transmits the weighted sum of independent radar waveforms and communication symbols, forming multiple beams towards the radar targets and the communication receivers, respectively. The design of the weighting coefficients is formulated as an optimization problem whose objective is the performance of the MIMO radar transmit beamforming, while guaranteeing that the signal-to-interference-plus-noise ratio (SINR) at each communication user is higher than a given threshold. Despite the non-convexity of the proposed optimization problem, we prove that it can be relaxed into a convex one, where the relaxation is tight. We then propose a reduced complexity design based on zero-forcing the inter-user interference and radar interference. Unlike previous works, which focused on the transmission of communication symbols to synthesize a radar transmit beam pattern, our method provides more degrees of freedom for MIMO radar and is thus able to obtain improved radar performance, as demonstrated in our simulation study. Furthermore, the proposed dual-function scheme approaches the radar performance of the radar-only scheme, i.e., without spectrum sharing, under reasonable communication quality constraints.

Model-Driven Deep Learning for MIMO Detection

Abstract: In this paper, we investigate the model-driven deep learning (DL) for MIMO detection. In particular, the MIMO detector is specially designed by unfolding an iterative algorithm and adding some trainable parameters. Since the number of trainable parameters is much fewer than the data-driven DL based signal detector, the model-driven DL based MIMO detector can be rapidly trained with a much smaller data set. The proposed MIMO detector can be extended to soft-input soft-output detection easily. Furthermore, we investigate joint MIMO channel estimation and signal detection (JCESD), where the detector takes channel estimation error and channel statistics into consideration while channel estimation is refined by detected data and considers the detection error. Based on numerical results, the model-driven DL based MIMO detector significantly improves the performance of corresponding traditional iterative detector, outperforms other DL-based MIMO detectors and exhibits superior robustness to various mismatches.

Intelligent Reflecting Surface Aided Multigroup Multicast MISO Communication Systems

Abstract: Intelligent reflecting surface (IRS) has recently been envisioned to offer unprecedented massive multiple-input multiple-output (MIMO)-like gains by deploying large-scale and low-cost passive reflection elements. By adjusting the reflection coefficients, the IRS can change the phase shifts on the impinging electromagnetic waves so that it can smartly reconfigure the signal propagation environment and enhance the power of the desired received signal or suppress the interference signal. In this paper, we consider downlink multigroup multicast communication systems assisted by an IRS. We aim for maximizing the sum rate of all the multicasting groups by the joint optimization of the precoding matrix at the base station (BS) and the reflection coefficients at the IRS under both the power and unit-modulus constraint. To tackle this non-convex problem, we propose two efficient algorithms under the majorization–minimization (MM) algorithm framework. Specifically, a concave lower bound surrogate objective function of each user's rate has been derived firstly, based on which two sets of variables can be updated alternately by solving two corresponding second-order cone programming (SOCP) problems. Then, in order to reduce the computational complexity, we derive another concave lower bound function of each group's rate for each set of variables at every iteration, and obtain the closed-form solutions under these loose surrogate objective functions. Finally, the simulation results demonstrate the benefits in terms of the spectral and energy efficiency of the introduced IRS and the effectiveness in terms of the convergence and complexity of our proposed algorithms.

Compressive Sensing-Based Adaptive Active User Detection and Channel Estimation: Massive Access Meets Massive MIMO

Abstract: This paper considers massive access in massive multiple-input multiple-output (MIMO) systems and proposes an adaptive active user detection and channel estimation scheme based on compressive sensing. By exploiting the sporadic traffic of massive connected user equipments and the virtual angular domain sparsity of massive MIMO channels, the proposed scheme can support massive access with dramatically reduced access latency. Specifically, we design non-orthogonal pseudo-random pilots for uplink broadband massive access, and formulate the active user detection and channel estimation as a generalized multiple measurement vector compressive sensing problem. Furthermore, by leveraging the structured sparsity of the uplink channel matrix, we propose an efficient generalized multiple measurement vector approximate message passing (GMMV-AMP) algorithm to realize joint active user detection and channel estimation based on a spatial domain or an angular domain channel model. To jointly exploit the channel sparsity present in both the spatial and the angular domains for enhanced performance, a Turbo-GMMV-AMP algorithm is developed for detecting the active users and estimating their channels in an alternating manner. Finally, an adaptive access scheme is proposed, which adapts the access latency to guarantee reliable massive access for practical systems with unknown channel sparsity level. Additionally, the state evolution of the proposed GMMV-AMP algorithm is derived to predict its performance. Simulation results demonstrate the superiority of the proposed active user detection and channel estimation schemes compared to several baseline schemes.

Optimal Wireless Resource Allocation With Random Edge Graph Neural Networks

Abstract: We consider the problem of optimally allocating resources across a set of transmitters and receivers in a wireless network. The resulting optimization problem takes the form of constrained statistical learning, in which solutions can be found in a model-free manner by parameterizing the resource allocation policy. Convolutional neural networks architectures are an attractive option for parameterization, as their dimensionality is small and does not scale with network size. We introduce the random edge graph neural network (REGNN), which performs convolutions over random graphs formed by the fading interference patterns in the wireless network. The REGNN-based allocation policies are shown to retain an important permutation equivariance property that makes them amenable to transference to different networks. We further present an unsupervised model-free primal-dual learning algorithm to train the weights of the REGNN. Through numerical simulations, we demonstrate the strong performance REGNNs obtain relative to heuristic benchmarks and their transference capabilities.

Stability Properties of Graph Neural Networks

Abstract: Graph neural networks (GNNs) have emerged as a powerful tool for nonlinear processing of graph signals, exhibiting success in recommender systems, power outage prediction, and motion planning, among others. GNNs consist of a cascade of layers, each of which applies a graph convolution, followed by a pointwise nonlinearity. In this work, we study the impact that changes in the underlying topology have on the output of the GNN. First, we show that GNNs are permutation equivariant, which implies that they effectively exploit internal symmetries of the underlying topology. Then, we prove that graph convolutions with integral Lipschitz filters, in combination with the frequency mixing effect of the corresponding nonlinearities, yields an architecture that is both stable to small changes in the underlying topology, and discriminative of information located at high frequencies. These are two properties that cannot simultaneously hold when using only linear graph filters, which are either discriminative or stable, thus explaining the superior performance of GNNs.

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

Abstract: We consider a distributed learning problem over multiple access channel (MAC) using a large wireless network. The computation is made by the network edge and is based on received data from a large number of distributed nodes which transmit over a noisy fading MAC. The objective function is a sum of the nodes’ local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning. We develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the distributed learning problem over MAC. Specifically, the nodes transmit an analog function of the local gradient using common shaping waveforms and the network edge receives a superposition of the analog transmitted signals used for updating the estimate. GBMA does not require power control or beamforming to cancel the fading effect as in other algorithms, and operates directly with noisy distorted gradients. We analyze the performance of GBMA theoretically, and prove that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks. Specifically, we establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. Furthermore, we provide energy scaling laws for approaching the centralized convergence rate as the number of nodes increases. Finally, experimental results support the theoretical findings, and demonstrate strong performance of GBMA using synthetic and real data.

Topological Signal Processing Over Simplicial Complexes

Abstract: The goal of this paper is to establish the fundamental tools to analyze signals defined over a topological space, i.e. a set of points along with a set of neighborhood relations. This setup does not require the definition of a metric and then it is especially useful to deal with signals defined over non-metric spaces. We focus on signals defined over simplicial complexes. Graph Signal Processing (GSP) represents a special case of Topological Signal Processing (TSP), referring to the situation where the signals are associated only with the vertices of a graph. Even though the theory can be applied to signals of any order, we focus on signals defined over the edges of a graph and show how building a simplicial complex of order two, i.e. including triangles, yields benefits in the analysis of edge signals. After reviewing the basic principles of algebraic topology, we derive a sampling theory for signals of any order and emphasize the interplay between signals of different order. Then we propose a method to infer the topology of a simplicial complex from data. We conclude with applications to traffic analysis over wireless networks and to the processing of discrete vector fields to illustrate the benefits of the proposed methodologies.

MAJoRCom: A Dual-Function Radar Communication System Using Index Modulation

Abstract: Dual-function radar communication (DFRC) systems implement both sensing and communication using the same hardware. Such schemes are often more efficient in terms of size, power, and cost, over using distinct radar and communication systems. Since these functionalities share resources such as spectrum, power, and antennas, DFRC methods typically entail some degradation in both radar and communication performance. In this work we propose a DFRC scheme based on the carrier agile phased array radar (CAESAR), which combines frequency and spatial agility. The proposed DFRC system, referred to as multi-carrier agile joint radar communication (MAJoRCom), exploits the inherent spatial and spectral randomness of CAESAR to convey digital messages in the form of index modulation. The resulting communication scheme naturally coexists with the radar functionality, and thus does not come at the cost of reduced radar performance. We analyze the performance of MAJoRCom, quantifying its achievable bit rate. In addition, we develop a low complexity decoder and a codebook design approach, which simplify the recovery of the communicated bits. Our numerical results demonstrate that MAJoRCom is capable of achieving a bit rate which is comparable to utilizing independent communication modules without affecting the radar performance, and that our proposed low-complexity decoder allows the receiver to reliably recover the transmitted symbols with an affordable computational burden.

Adaptive Virtual Waveform Design for Millimeter-Wave Joint Communication–Radar

Abstract: Joint communication and radar (JCR) waveforms with fully digital baseband generation and processing can now be realized at the millimeter-wave (mmWave) band. Prior work has developed a mmWave wireless local area network (WLAN)-based JCR that exploits the WLAN preamble for radars. The performance of target velocity estimation, however, was limited. In this paper, we propose a virtual waveform design for an adaptive mmWave JCR. The proposed system transmits a few non-uniformly placed preambles to construct several receive virtual preambles for enhancing velocity estimation accuracy, at the cost of only a small reduction in the communication data rate. We evaluate JCR performance trade-offs using the Cramer-Rao Bound (CRB) metric for radar estimation and a novel distortion minimum mean square error (MMSE) metric for data communication. Additionally, we develop three different MMSE-based optimization problems for the adaptive JCR waveform design. Simulations show that an optimal virtual (non-uniform) waveform achieves a significant performance improvement as compared to a uniform waveform. For a radar CRB constrained optimization, the optimal radar range of operation and the optimal communication distortion MMSE (DMMSE) are improved. For a communication DMMSE constrained optimization with a high DMMSE constraint, the optimal radar CRB is enhanced. For a weighted MMSE average optimization, the advantage of the virtual waveform over the uniform waveform is increased with decreased communication weighting. Comparison of MMSE-based optimization with traditional virtual preamble count-based optimization indicated that the conventional solution converges to the MMSE-based one only for a small number of targets and a high signal-to-noise ratio.

Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications

Abstract: The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most existing theory for this class has been mainly developed for problems with certain special convex-concave structure. Therefore, it cannot be used to guide the algorithm design for many interesting problems in SPCOM, where various kinds of non-convexity arise. In this work, we consider a block-wise one-sided non-convex min-max problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is (strongly) concave. We propose a class of simple algorithms named Hybrid Block Successive Approximation (HiBSA), which alternatingly performs gradient descent-type steps for the minimization blocks and gradient ascent-type steps for the maximization problem. A key element in the proposed algorithm is the use of certain regularization and penalty sequences, which stabilize the algorithm and ensure convergence. We show that HiBSA converges to some properly defined first-order stationary solutions with quantifiable global rates. To validate the efficiency of the proposed algorithms, we conduct numerical tests on a number of problems, including the robust learning problem, the non-convex min-utility maximization problems, and certain wireless jamming problem arising in interfering channels.

A Solution for Large-Scale Multi-Object Tracking

Abstract: A large-scale multi-object tracker based on the generalised labeled multi-Bernoulli (GLMB) filter is proposed. The algorithm is capable of tracking a very large, unknown and time-varying number of objects simultaneously, in the presence of a high number of false alarms, as well as missed detections and measurement origin uncertainty due to closely spaced objects. The algorithm is demonstrated on a simulated tracking scenario, where the peak number objects appearing simultaneously exceeds one million. Additionally, we introduce a new method of applying the optimal sub-pattern assignment (OSPA) metric to determine a meaningful distance between two sets of tracks. We also develop an efficient strategy for its exact computation in large-scale scenarios to evaluate the performance of the proposed tracker.

Penalty Dual Decomposition Method for Nonsmooth Nonconvex Optimization—Part I: Algorithms and Convergence Analysis

Abstract: Many contemporary signal processing, machine learning and wireless communication applications can be formulated as nonconvex nonsmooth optimization problems. Often there is a lack of efficient algorithms for these problems, especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this work, we propose an algorithm named penalty dual decomposition (PDD) for these difficult problems and discuss its various applications. The PDD is a double-loop iterative algorithm. Its inner iteration is used to inexactly solve a nonconvex nonsmooth augmented Lagrangian problem via block-coordinate-descent-type methods, while its outer iteration updates the dual variables and/or a penalty parameter. In Part I of this work, we describe the PDD algorithm and establish its convergence to KKT solutions. In Part II we evaluate the performance of PDD by customizing it to three applications arising from signal processing and wireless communications.

Resource Scheduling for Distributed Multi-Target Tracking in Netted Colocated MIMO Radar Systems

Abstract: In this paper, an effective solution is proposed for joint beam and power scheduling (JBPS) in the netted Colocated MIMO (C-MIMO) radar systems for distributed multi-target tracking (MTT). At its core, the proposed solution includes a distributed fusion architecture that reduces the communication requirements while maintaining the overall robustness of the system. The distributed fusion architecture employs the covariance intersection (CI) fusion to address the unknown information correlations among radar nodes. Each C-MIMO radar node in the network can generate a time-varying number of beams with controllable transmitting power by waveform synthesis, thus is capable of accomplishing multiple tracking tasks simultaneously. To maximize the global MTT performance of the radar network, the proposed JBPS solution implements an online resource scheduling, regarding both the generated beams and the transmitted power of all radar nodes, based on the feedback of the MTT results. A scaled accuracy-based objective function is designed to quantify the global MTT performance while properly taking into account different target priorities on resource allocation. The Bayesian Cramer-Rao lower bound (BCRLB) for CI fusion rule is derived and utilized as the constituent of the objective function since it provides a lower bound on the accuracy of the target state estimates. As the formulated JBPS problem is non-convex, we propose a fast reward-based iterative descending approach to solve it effectively. Numerical results show that the proposed JBPS can deliver superior performance in terms of maximizing the overall MTT performance while possessing high flexibility on the resource allocation regarding different target priorities.

Optimal Resource Allocation for Asynchronous Multiple Targets Tracking in Heterogeneous Radar Networks

Abstract: In this paper, two optimal resource allocation schemes are developed for asynchronous multiple targets tracking (MTT) in heterogeneous radar networks The key idea of heterogeneous resource allocation (HRA) schemes is to coordinate the heterogeneous transmit resource (transmit power, dwell time, etc) of different types of radars to achieve a better resource utilization efficiency We use the Bayesian Cramer-Rao lower bound (BCRLB) as a metric function to quantify the target tracking performance and build the following two HRA schemes: For a given system resource budget: (1) Minimize the total resource consumption for the given BCRLB requirements on multiple targets and (2) maximize the overall MTT accuracy Instead of updating the state of each target recursively at different measurement arrival times, we combine multiple asynchronous measurements into a single composite measurement and use it as an input of the tracking filter for state estimation In such a case, target tracking BCRLB no longer needs to be recursively calculated, and thus, we can formulate the HRA schemes as two convex optimization problems We subsequently design two efficient methods to solve these problems by exploring their unique structures Simulation results demonstrate that the HRA processes can either provide a smaller overall MTT BCRLB for given resource budgets or require fewer resources to establish the same tracking performance for multiple targets

Multi-UAV Interference Coordination via Joint Trajectory and Power Control

Abstract: Recently, unmanned aerial vehicles (UAVs) have found growing applications in wireless communications and sensor networks. One of the key challenges for UAV-based wireless networks lies in managing the strong cross-link interference caused by the line-of-sight dominated propagation conditions. In this article, we address this challenge by studying a UAV-enabled interference channel (UAV-IC), where each of the $K$ UAVs communicates with its associated ground terminal. To exploit the new degree of freedom of UAV mobility, we formulate a joint trajectory and power control (TPC) problem for maximizing the aggregate sum rate of the UAV-IC for a given flight interval, under practical constraints on the UAV flying speed, altitude, and collision avoidance. These constraints couple the TPC variables across different time slots and UAVs, leading to a challenging large-scale and non-convex optimization problem. We show that the optimal TPC solution follows the fly--hover--fly strategy, based on which the problem can be handled first by finding optimal hovering locations followed by solving a dimension-reduced TPC problem. For the reduced TPC problem, we propose a successive convex approximation algorithm. To further reduce the computation time, we develop a parallel TPC algorithm that is efficiently implementable over multi-core CPUs. We also propose a segment-by-segment method that decomposes the TPC problem into sequential TPC subproblems each with a smaller problem dimension. Simulation results demonstrate the superior computation time efficiency of the proposed algorithms, and also show that the UAV-IC can yield higher network sum rate than the benchmark orthogonal schemes.

A Dimension Reduction-Based Joint Activity Detection and Channel Estimation Algorithm for Massive Access

Abstract: Grant-free random access is a promising protocol to support massive access in beyond fifth-generation (B5G) cellular Internet-of-Things (IoT) with sporadic traffic. Specifically, in each coherence interval, the base station (BS) performs joint activity detection and channel estimation (JADCE) before data transmission. Due to the deployment of a large-scale antennas array and the existence of a huge number of IoT devices, JADCE usually has high computational complexity and needs long pilot sequences. To solve these challenges, this paper proposes a dimension reduction method, which projects the original device state matrix to a low-dimensional space by exploiting its sparse and low-rank structure. Then, we develop an optimized design framework with a coupled full column rank constraint for JADCE to reduce the size of the search space. However, the resulting problem is non-convex and highly intractable, for which the conventional convex relaxation approaches are inapplicable. To this end, we propose a logarithmic smoothing method for the non-smoothed objective function and transform the interested matrix to a positive semidefinite matrix, followed by giving a Riemannian trust-region algorithm to solve the problem in complex field. Simulation results show that the proposed algorithm is efficient to a large-scale JADCE problem and requires shorter pilot sequences than the state-of-art algorithms which only exploit the sparsity of device state matrix.

MIMO Radar Waveform Design in the Presence of Multiple Targets and Practical Constraints

Abstract: This paper deals with the joint design of Multiple-Input Multiple-Output (MIMO) radar transmit waveform and receive filter to enhance multiple targets detectability in the presence of signal-dependent (clutter) and independent disturbance. The worst-case Signal-to-Interference-Noise-Ratio (SINR) over multiple targets is explicitly maximized. To ensure hardware compatibility and the coexistence between MIMO radar and other wireless systems, constant modulus and spectral restrictions on the waveform are incorporated in our design. A max-min non-convex optimization problem emerges as a function of the transmit waveform, which we solve via a novel polynomial-time iterative procedure that involves solving a sequence of convex problems with constraints that evolve with every iteration. The overall algorithm follows an alternate optimization over the receive filter and transmit waveform. For the problem of waveform optimization (which is our central contribution), we provide analytical guarantees of monotonic cost function improvement with proof of convergence to a solution that satisfies the KarushKuhnTucker (KKT) conditions. We also develop extensions that address the well-known waveform similarity constraint. By simulating challenging practical scenarios, we evaluate the proposed algorithm against the state-of-the-art methods in terms of the achieved SINR value and the computational complexity. Overall, we show that our proposal outperforms state of the art competing methods while providing the most favorable performance-complexity balance.

Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks

Abstract: This paper deals with distributed finite-sum optimization for learning over multiple workers in the presence of malicious Byzantine attacks. Most resilient approaches so far combine stochastic gradient descent (SGD) with different robust aggregation rules. However, the sizeable SGD-induced stochastic gradient noise challenges discerning malicious messages sent by the Byzantine attackers from noisy stochastic gradients sent by the ‘honest’ workers. This motivates reducing the variance of stochastic gradients as a means of robustifying SGD. To this end, a novel Byzantine attack resilient distributed (Byrd-) SAGA approach is introduced for federated learning tasks involving multiple workers. Rather than the mean employed by distributed SAGA, the novel Byrd-SAGA relies on the geometric median to aggregate the corrected stochastic gradients sent by the workers. When less than half of the workers are Byzantine attackers, Byrd-SAGA attains provably linear convergence to a neighborhood of the optimal solution, with the asymptotic learning error determined by the number of Byzantine workers. Numerical tests corroborate the robustness to various Byzantine attacks, as well as the merits of Byrd-SAGA over Byzantine attack resilient distributed SGD.

Variance-Reduced Decentralized Stochastic Optimization With Accelerated Convergence

Abstract: This paper describes a novel algorithmic framework to minimize a finite-sum of functions available over a network of nodes. The proposed framework, that we call GT-VR , is stochastic and decentralized, and thus is particularly suitable for problems where large-scale, potentially private data, cannot be collected or processed at a centralized server. The GT-VR framework leads to a family of algorithms with two key ingredients: (i) local variance reduction , that enables estimating the local batch gradients from arbitrarily drawn samples of local data; and, (ii) global gradient tracking , which fuses the gradient information across the nodes. Naturally, combining different variance reduction and gradient tracking techniques leads to different algorithms of interest with valuable practical tradeoffs and design considerations. Our focus in this paper is on two instantiations of the ${\bf \mathtt {GT-VR}}$ framework, namely GT-SAGA and GT-SVRG , that, similar to their centralized counterparts ( SAGA and SVRG ), exhibit a compromise between space and time. We show that both GT-SAGA and GT-SVRG achieve accelerated linear convergence for smooth and strongly convex problems and further describe the regimes in which they achieve non-asymptotic, network-independent linear convergence rates that are faster with respect to the existing decentralized first-order schemes. Moreover, we show that both algorithms achieve a linear speedup in such regimes compared to their centralized counterparts that process all data at a single node. Extensive simulations illustrate the convergence behavior of the corresponding algorithms.

Gated Graph Recurrent Neural Networks

Abstract: Graph processes exhibit a temporal structure determined by the sequence index and and a spatial structure determined by the graph support. To learn from graph processes, an information processing architecture must then be able to exploit both underlying structures. We introduce Graph Recurrent Neural Networks (GRNNs) as a general learning framework that achieves this goal by leveraging the notion of a recurrent hidden state together with graph signal processing (GSP). In the GRNN, the number of learnable parameters is independent of the length of the sequence and of the size of the graph, guaranteeing scalability. We prove that GRNNs are permutation equivariant and that they are stable to perturbations of the underlying graph support. To address the problem of vanishing gradients, we also put forward gated GRNNs with three different gating mechanisms: time, node and edge gates. In numerical experiments involving both synthetic and real datasets, time-gated GRNNs are shown to improve upon GRNNs in problems with long term dependencies, while node and edge gates help encode long range dependencies present in the graph. The numerical results also show that GRNNs outperform GNNs and RNNs, highlighting the importance of taking both the temporal and graph structures of a graph process into account.

On the Design of Multi-Spectrally Constrained Constant Modulus Radar Signals

Abstract: This paper deals with the synthesis of constant modulus waveforms that optimize radar performance while satisfying multiple spectral compatibility constraints. For each shared band, a precise control is imposed on the injected interference energy. Furthermore, the compliance with amplifiers operating in saturation is ensured at the design stage where phase-only waveforms are considered. To tackle the resulting NP-hard optimization problem, an iterative procedure based on the coordinate descent method is introduced. The overall computational burden of the algorithm is linear with respect to the code length as well as the number of iterations and less then cubic with reference to the number of spectral constraints. Hence, some case studies are reported to highlight the effectiveness of the technique.

Distributed Online Convex Optimization With Time-Varying Coupled Inequality Constraints

Abstract: This paper considers distributed online optimization with time-varying coupled inequality constraints. The global objective function is composed of local convex cost and regularization functions and the coupled constraint function is the sum of local convex functions. A distributed online primal-dual dynamic mirror descent algorithm is proposed to solve this problem, where the local cost, regularization, and constraint functions are held privately and revealed only after each time slot. Without assuming Slater's condition, we first derive regret and constraint violation bounds for the algorithm and show how they depend on the stepsize sequences, the accumulated dynamic variation of the comparator sequence, the number of agents, and the network connectivity. As a result, under some natural decreasing stepsize sequences, we prove that the algorithm achieves sublinear dynamic regret and constraint violation if the accumulated dynamic variation of the optimal sequence also grows sublinearly. We also prove that the algorithm achieves sublinear static regret and constraint violation under mild conditions. Assuming Slater's condition, we show that the algorithm achieves smaller bounds on the constraint violation. In addition, smaller bounds on the static regret are achieved when the objective function is strongly convex. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Feature Graph Learning for 3D Point Cloud Denoising

Abstract: Identifying an appropriate underlying graph kernel that reflects pairwise similarities is critical in many recent graph spectral signal restoration schemes, including image denoising, dequantization, and contrast enhancement. Existing graph learning algorithms compute the most likely entries of a properly defined graph Laplacian matrix $\mathbf {L}$ , but require a large number of signal observations $\mathbf {z}$ 's for a stable estimate. In this work, we assume instead the availability of a relevant feature vector $\mathbf {f}_i$ per node $i$ , from which we compute an optimal feature graph via optimization of a feature metric. Specifically, we alternately optimize the diagonal and off-diagonal entries of a Mahalanobis distance matrix $\mathbf {M}$ by minimizing the graph Laplacian regularizer (GLR) $\mathbf {z}^{\top } \mathbf {L} \mathbf {z}$ , where edge weight is $w_{i,j} = \exp \lbrace - (\mathbf {f}_i - \mathbf {f}_j)^{\top } \mathbf {M} (\mathbf {f}_i - \mathbf {f}_j) \rbrace$ , given a single observation $\mathbf {z}$ . We optimize diagonal entries via proximal gradient (PG), where we constrain $\mathbf {M}$ to be positive definite (PD) via linear inequalities derived from the Gershgorin circle theorem. To optimize off-diagonal entries, we design a block descent algorithm that iteratively optimizes one row and column of $\mathbf {M}$ . To keep $\mathbf {M}$ PD, we constrain the Schur complement of sub-matrix $\mathbf {M}_{2,2}$ of $\mathbf {M}$ to be PD when optimizing via PG. Our algorithm mitigates full eigen-decomposition of $\mathbf {M}$ , thus ensuring fast computation speed even when feature vector $\mathbf {f}_i$ has high dimension. To validate its usefulness, we apply our feature graph learning algorithm to the problem of 3D point cloud denoising, resulting in state-of-the-art performance compared to competing schemes in extensive experiments.

Massive MIMO Radar for Target Detection

Abstract: Since the seminal paper by Marzetta from 2010, the Massive MIMO paradigm in communication systems has changed from being a theoretical scaled-up version of MIMO, with an infinite number of antennas, to a practical technology. Its key concepts have been adopted in the 5G new radio standard and base stations, where 64 fully-digital transceivers have been commercially deployed. Motivated by these recent developments, this paper considers a co-located MIMO radar with $M_T$ transmitting and $M_R$ receiving antennas and explores the potential benefits of having a large number of virtual spatial antenna channels $N=M_TM_R$ . Particularly, we focus on the target detection problem and develop a robust Wald-type test that guarantees certain detection performance, regardless of the unknown statistical characterization of the disturbance. Closed-form expressions for the probabilities of false alarm and detection are derived for the asymptotic regime $N\rightarrow \infty$ . Numerical results are used to validate the asymptotic analysis in the finite system regime with different disturbance models. Our results imply that there always exists a sufficient number of antennas for which the performance requirements are satisfied, without any a-priori knowledge of the disturbance statistics. This is referred to as the Massive MIMO regime of the radar system.

Two-Dimensional Modular Chaotification System for Improving Chaos Complexity

Abstract: Chaotic systems are widely studied in various research areas such as signal processing and secure communication. Existing chaotic systems may have drawbacks such as discontinuous chaotic ranges and incomplete output distributions. These drawbacks may lead to the defects of some chaos-based applications. To accommodate these challenges, this paper proposes a two-dimensional (2D) modular chaotification system (2D-MCS) to improve the chaos complexity of any 2D chaotic map. Because the modular operation is a bounded transform, the improved chaotic maps by 2D-MCS can generate chaotic behaviors in wide parameter ranges while existing chaotic maps cannot. Three improved chaotic maps are presented as typical examples to verify the effectiveness of 2D-MCS. The chaos properties of one example of 2D-MCS are mathematically analyzed using the definition of Lyapunov exponent. Performance evaluations demonstrate that these improved chaotic maps have continuous and large chaotic ranges, and their outputs are distributed more uniformly than the outputs of existing 2D chaotic maps. To show the application of 2D-MCS, we apply the improved chaotic maps of 2D-MCS to secure communication. The simulation results show that these improved chaotic maps exhibit better performance than several existing and newly developed chaotic maps in terms of resisting different channel noise.

New Viewpoint and Algorithms for Water-Filling Solutions in Wireless Communications

Abstract: Water-filling solutions play an important role in the designs for wireless communications, e.g., transmit covariance matrix design. A traditional physical understanding is to use the analogy of pouring water over a pool with fluctuating bottom. Numerous variants of water-filling solutions have been discovered during the evolution of wireless networks. To obtain the solution values, iterative computations are required, even for simple cases with compact mathematical formulations. Thus, algorithm design is a key issue for the practical use of water-filling solutions, which however has been given marginal attention in the literature. Many existing algorithms are designed on a case-by-case basis for the variations of water-filling solutions and/or with complex logics. In this paper, a new viewpoint for water-filling solutions is proposed to understand the problem dynamically by considering changes in the increasing rates on different subchannels. This fresh viewpoint provides a useful mechanism and fundamental information for finding the optimization solution values. Based on this new understanding, a novel and comprehensive method for practical water-filling algorithm design is proposed, which can be used for systems with various performance metrics and power constraints, even for systems with imperfect channel state information (CSI).

On Arithmetic Average Fusion and Its Application for Distributed Multi-Bernoulli Multitarget Tracking

Abstract: This paper addresses the problem of distributed multitarget detection and tracking based on the linear arithmetic average (AA) fusion. We first analyze the conservativeness and Frechet mean properties of the AA fusion, presenting new analyses based on a literature review. Second, we propose a target-wise fusion rule for tailoring the AA fusion to accommodate the multi-Bernoulli (MB) process, in which only significant Bernoulli components, each represented by an individual Gaussian mixture, are disseminated and fused in a Bernoulli-to-Bernoulli (B2B) manner. For internode communication, both the consensus and flooding schemes are investigated, respectively. At the core of the proposed MB fusion algorithms, Bernoulli components obtained at different sensors are associated via either clustering or pairwise assignment so that the MB fusion problem is decomposed to parallel B2B fusion subproblems, each resolved via exact Bernoulli-AA fusion. Third, two communicatively and computationally efficient cardinality consensus approaches are presented which merely disseminate and fuse target existence probabilities among local MB filters. Finally, the performance of these four approaches in terms of accuracy, computing efficiency, and communication cost is tested in two simulation scenarios.