scispace - formally typeset
Search or ask a question

Showing papers in "IEEE Transactions on Information Theory in 2009"


Journal ArticleDOI
Erdal Arikan1
TL;DR: The paper proves that, given any B-DMC W with I(W) > 0 and any target rate R< I( W) there exists a sequence of polar codes {Cfrn;nges1} such that Cfrn has block-length N=2n, rate ges R, and probability of block error under successive cancellation decoding bounded as Pe(N,R) les O(N-1/4) independently of the code rate.
Abstract: A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity I(W) of any given binary-input discrete memoryless channel (B-DMC) W. The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of N independent copies of a given B-DMC W, a second set of N binary-input channels {WN(i)1 les i les N} such that, as N becomes large, the fraction of indices i for which I(WN(i)) is near 1 approaches I(W) and the fraction for which I(WN(i)) is near 0 approaches 1-I(W). The polarized channels {WN(i)} are well-conditioned for channel coding: one need only send data at rate 1 through those with capacity near 1 and at rate 0 through the remaining. Codes constructed on the basis of this idea are called polar codes. The paper proves that, given any B-DMC W with I(W) > 0 and any target rate R< I(W) there exists a sequence of polar codes {Cfrn;nges1} such that Cfrn has block-length N=2n , rate ges R, and probability of block error under successive cancellation decoding bounded as Pe(N,R) les O(N-1/4) independently of the code rate. This performance is achievable by encoders and decoders with complexity O(N logN) for each.

3,554 citations


Journal ArticleDOI
TL;DR: The presented analysis shows that in the noiseless setting, the proposed algorithm can exactly reconstruct arbitrary sparse signals provided that the sensing matrix satisfies the restricted isometry property with a constant parameter.
Abstract: We propose a new method for reconstruction of sparse signals with and without noisy perturbations, termed the subspace pursuit algorithm. The algorithm has two important characteristics: low computational complexity, comparable to that of orthogonal matching pursuit techniques when applied to very sparse signals, and reconstruction accuracy of the same order as that of linear programming (LP) optimization methods. The presented analysis shows that in the noiseless setting, the proposed algorithm can exactly reconstruct arbitrary sparse signals provided that the sensing matrix satisfies the restricted isometry property with a constant parameter. In the noisy setting and in the case that the signal is not exactly sparse, it can be shown that the mean-squared error of the reconstruction is upper-bounded by constant multiples of the measurement and signal perturbation energies.

2,235 citations


Journal ArticleDOI
TL;DR: This work analyzes the behavior of l1-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern of a vector beta* based on observations contaminated by noise, and establishes precise conditions on the problem dimension p, the number k of nonzero elements in beta*, and the number of observations n.
Abstract: The problem of consistently estimating the sparsity pattern of a vector beta* isin Rp based on observations contaminated by noise arises in various contexts, including signal denoising, sparse approximation, compressed sensing, and model selection. We analyze the behavior of l1-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern. Our main result is to establish precise conditions on the problem dimension p, the number k of nonzero elements in beta*, and the number of observations n that are necessary and sufficient for sparsity pattern recovery using the Lasso. We first analyze the case of observations made using deterministic design matrices and sub-Gaussian additive noise, and provide sufficient conditions for support recovery and linfin-error bounds, as well as results showing the necessity of incoherence and bounds on the minimum value. We then turn to the case of random designs, in which each row of the design is drawn from a N (0, Sigma) ensemble. For a broad class of Gaussian ensembles satisfying mutual incoherence conditions, we compute explicit values of thresholds 0 0, if n > 2 (thetasu + delta) klog (p- k), then the Lasso succeeds in recovering the sparsity pattern with probability converging to one for large problems, whereas for n < 2 (thetasl - delta)klog (p - k), then the probability of successful recovery converges to zero. For the special case of the uniform Gaussian ensemble (Sigma = Iptimesp), we show that thetasl = thetas

1,438 citations


Journal ArticleDOI
TL;DR: This paper develops a general framework for robust and efficient recovery of nonlinear but structured signal models, in which x lies in a union of subspaces, and presents an equivalence condition under which the proposed convex algorithm is guaranteed to recover the original signal.
Abstract: Traditional sampling theories consider the problem of reconstructing an unknown signal x from a series of samples. A prevalent assumption which often guarantees recovery from the given measurements is that x lies in a known subspace. Recently, there has been growing interest in nonlinear but structured signal models, in which x lies in a union of subspaces. In this paper, we develop a general framework for robust and efficient recovery of such signals from a given set of samples. More specifically, we treat the case in which x lies in a sum of k subspaces, chosen from a larger set of m possibilities. The samples are modeled as inner products with an arbitrary set of sampling functions. To derive an efficient and robust recovery algorithm, we show that our problem can be formulated as that of recovering a block-sparse vector whose nonzero elements appear in fixed blocks. We then propose a mixed lscr2/lscr1 program for block sparse recovery. Our main result is an equivalence condition under which the proposed convex algorithm is guaranteed to recover the original signal. This result relies on the notion of block restricted isometry property (RIP), which is a generalization of the standard RIP used extensively in the context of compressed sensing. Based on RIP, we also prove stability of our approach in the presence of noise and modeling errors. A special case of our framework is that of recovering multiple measurement vectors (MMV) that share a joint sparsity pattern. Adapting our results to this context leads to new MMV recovery methods as well as equivalence conditions under which the entire set can be determined efficiently.

966 citations


Journal ArticleDOI
TL;DR: The results establish a direct connection between min- and max-entropies, known to characterize information-processing tasks such as randomness extraction and state merging, and basic operational problems.
Abstract: In this paper, we show that the conditional min-entropy H min(A |B) of a bipartite state rhoAB is directly related to the maximum achievable overlap with a maximally entangled state if only local actions on the B-part of rhoAB are allowed. In the special case where A is classical, this overlap corresponds to the probability of guessing A given B. In a similar vein, we connect the conditional max-entropy H max(A |B) to the maximum fidelity of rhoAB with a product state that is completely mixed on A. In the case where A is classical, this corresponds to the security of A when used as a secret key in the presence of an adversary holding B. Because min- and max-entropies are known to characterize information-processing tasks such as randomness extraction and state merging, our results establish a direct connection between these tasks and basic operational problems. For example, they imply that the (logarithm of the) probability of guessing A given B is a lower bound on the number of uniform secret bits that can be extracted from A relative to an adversary holding B.

692 citations


Journal ArticleDOI
TL;DR: Several commonly-used sparsity measures are compared based on whether or not they satisfy these six propositions and only two of these measures satisfy all six: the pq-mean with p les 1, q > 1 and the Gini index.
Abstract: Sparsity of representations of signals has been shown to be a key concept of fundamental importance in fields such as blind source separation, compression, sampling and signal analysis. The aim of this paper is to compare several commonly-used sparsity measures based on intuitive attributes. Intuitively, a sparse representation is one in which a small number of coefficients contain a large proportion of the energy. In this paper, six properties are discussed: (Robin Hood, Scaling, Rising Tide, Cloning, Bill Gates, and Babies), each of which a sparsity measure should have. The main contributions of this paper are the proofs and the associated summary table which classify commonly-used sparsity measures based on whether or not they satisfy these six propositions. Only two of these measures satisfy all six: the pq-mean with p les 1, q > 1 and the Gini index.

667 citations


Journal ArticleDOI
TL;DR: It is demonstrated that, in a ldquohigh-interferencerdquo regime, multiuser decoding at the primary receiver is optimal from the standpoint of maximal jointly achievable rates for the primary and cognitive users.
Abstract: In this paper, we consider a communication scenario in which the primary and the cognitive radios wish to communicate to different receivers, subject to mutual interference. In the model that we use, the cognitive radio has noncausal knowledge of the primary radio's codeword. We characterize the largest rate at which the cognitive radio can reliably communicate under the constraint that 1) no rate degradation is created for the primary user, and 2) the primary receiver uses a single-user decoder just as it would in the absence of the cognitive radio. The result holds in a ldquolow-interferencerdquo regime in which the cognitive radio is closer to its receiver than to the primary receiver. In this regime, our results are subsumed by the results derived in a concurrent and independent work (Wu , 2007). We also demonstrate that, in a ldquohigh-interferencerdquo regime, multiuser decoding at the primary receiver is optimal from the standpoint of maximal jointly achievable rates for the primary and cognitive users.

570 citations


Journal ArticleDOI
TL;DR: It is proved that the full Han-Kobayashi achievable rate region using Gaussian codebooks is equivalent to that of the one-sided Gaussian IC for a particular range of channel parameters.
Abstract: The capacity region of the two-user Gaussian interference channel (IC) is studied. Three classes of channels are considered: weak, one-sided, and mixed Gaussian ICs. For the weak Gaussian IC, a new outer bound on the capacity region is obtained that outperforms previously known outer bounds. The sum capacity for a certain range of channel parameters is derived. For this range, it is proved that using Gaussian codebooks and treating interference as noise are optimal. It is shown that when Gaussian codebooks are used, the full Han-Kobayashi achievable rate region can be obtained by using the naive Han-Kobayashi achievable scheme over three frequency bands (equivalently, three subspaces). For the one-sided Gaussian IC, an alternative proof for the Sato's outer bound is presented. We derive the full Han-Kobayashi achievable rate region when Gaussian codebooks are utilized. For the mixed Gaussian IC, a new outer bound is obtained that outperforms previously known outer bounds. For this case, the sum capacity for the entire range of channel parameters is derived. It is proved that the full Han-Kobayashi achievable rate region using Gaussian codebooks is equivalent to that of the one-sided Gaussian IC for a particular range of channel parameters.

567 citations


Journal ArticleDOI
TL;DR: It is shown that if the channel crosstalk coefficient magnitudes lie below thresholds defined by the power constraints then single-user detection at each receiver is sum-rate optimal, i.e., treating the interference as noise incurs no loss in performance.
Abstract: A new outer bound on the capacity region of Gaussian interference channels is developed. The bound combines and improves existing genie-aided methods and is shown to give the sum-rate capacity for noisy interference as defined in this paper. Specifically, it is shown that if the channel crosstalk coefficient magnitudes lie below thresholds defined by the power constraints then single-user detection at each receiver is sum-rate optimal, i.e., treating the interference as noise incurs no loss in performance. This is the first capacity result for the Gaussian interference channel with weak to moderate interference. Furthermore, for certain mixed (weak and strong) interference scenarios, the new outer bounds give a corner point of the capacity region.

536 citations


Journal ArticleDOI
TL;DR: The spectral divergence rates of the information spectrum approach are shown to be obtained from the smooth min- and max-relative entropies in the asymptotic limit.
Abstract: Two new relative entropy quantities, called the min- and max-relative entropies, are introduced and their properties are investigated. The well-known min- and max-entropies, introduced by Renner, are obtained from these. We define a new entanglement monotone, which we refer to as the max-relative entropy of entanglement, and which is an upper bound to the relative entropy of entanglement. We also generalize the min- and max-relative entropies to obtain smooth min-and max-relative entropies. These act as parent quantities for the smooth Renyi entropies (ETH Zurich, Ph.D. dissertation, 2005), and allow us to define the analogues of the mutual information, in the smooth Renyi entropy framework. Further, the spectral divergence rates of the information spectrum approach are shown to be obtained from the smooth min- and max-relative entropies in the asymptotic limit.

528 citations


Journal ArticleDOI
TL;DR: This paper presents an alternative characterization of the secrecy capacity of the multiple-antenna wiretap channel under a more general matrix constraint on the channel input using a channel-enhancement argument.
Abstract: The secrecy capacity of the multiple-antenna wiretap channel under the average total power constraint was recently characterized, independently, by Khisti and Wornell and Oggier and Hassibi using a Sato-like argument and matrix analysis tools. This paper presents an alternative characterization of the secrecy capacity of the multiple-antenna wiretap channel under a more general matrix constraint on the channel input using a channel-enhancement argument. This characterization is by nature information-theoretic and is directly built on the intuition regarding to the optimal transmission strategy in this communication scenario.

Journal ArticleDOI
TL;DR: New, improved outer bounds on the capacity region are developed and it is shown that treating interference as noise achieves the sum capacity of the two-user Gaussian interference channel in a low-interference regime, where the interference parameters are below certain thresholds.
Abstract: Establishing the capacity region of a Gaussian interference network is an open problem in information theory. Recent progress on this problem has led to the characterization of the capacity region of a general two-user Gaussian interference channel within one bit. In this paper, we develop new, improved outer bounds on the capacity region. Using these bounds, we show that treating interference as noise achieves the sum capacity of the two-user Gaussian interference channel in a low-interference regime, where the interference parameters are below certain thresholds. We then generalize our techniques and results to Gaussian interference networks with more than two users. In particular, we demonstrate that the total interference threshold, below which treating interference as noise achieves the sum capacity, increases with the number of users.

Journal ArticleDOI
TL;DR: For a noisy linear observation model based on random measurement matrices drawn from general Gaussian measurementMatrices, this paper derives both a set of sufficient conditions for exact support recovery using an exhaustive search decoder, as well as aset of necessary conditions that any decoder must satisfy for exactSupport set recovery.
Abstract: The problem of sparsity pattern or support set recovery refers to estimating the set of nonzero coefficients of an unknown vector beta* isin Ropfp based on a set of n noisy observations. It arises in a variety of settings, including subset selection in regression, graphical model selection, signal denoising, compressive sensing, and constructive approximation. The sample complexity of a given method for subset recovery refers to the scaling of the required sample size n as a function of the signal dimension p, sparsity index k (number of non-zeroes in beta*), as well as the minimum value betamin of beta* over its support and other parameters of measurement matrix. This paper studies the information-theoretic limits of sparsity recovery: in particular, for a noisy linear observation model based on random measurement matrices drawn from general Gaussian measurement matrices, we derive both a set of sufficient conditions for exact support recovery using an exhaustive search decoder, as well as a set of necessary conditions that any decoder, regardless of its computational complexity, must satisfy for exact support recovery. This analysis of fundamental limits complements our previous work on sharp thresholds for support set recovery over the same set of random measurement ensembles using the polynomial-time Lasso method (lscr1-constrained quadratic programming).

Journal ArticleDOI
TL;DR: It is shown that the total number of degrees of freedom of the X network is equal to [(MN)/(M+N-1)] per orthogonal time and frequency dimension, and that the degrees offreedom outer bound is exactly achievable.
Abstract: We explore the degrees of freedom of M times N user wireless X networks, i.e., networks of M transmitters and N receivers where every transmitter has an independent message for every receiver. We derive a general outer bound on the degrees of freedom region of these networks. When all nodes have a single antenna and all channel coefficients vary in time or frequency, we show that the total number of degrees of freedom of the X network is equal to [(MN)/(M+N-1)] per orthogonal time and frequency dimension. Achievability is proved by constructing interference alignment schemes for X networks that can come arbitrarily close to the outer bound on degrees of freedom. For the case where either M=2 or N=2 we find that the degrees of freedom characterization also provides a capacity approximation that is accurate to within O(1) . For these cases the degrees of freedom outer bound is exactly achievable.

Journal ArticleDOI
TL;DR: This paper derives the distributional properties of the interference and provides upper and lower bounds for its distribution, and considers the probability of successful transmission in an interference-limited channel when fading is modeled as Rayleigh.
Abstract: In the analysis of large random wireless networks, the underlying node distribution is almost ubiquitously assumed to be the homogeneous Poisson point process. In this paper, the node locations are assumed to form a Poisson cluster process on the plane. We derive the distributional properties of the interference and provide upper and lower bounds for its distribution. We consider the probability of successful transmission in an interference-limited channel when fading is modeled as Rayleigh. We provide a numerically integrable expression for the outage probability and closed-form upper and lower bounds. We show that when the transmitter-receiver distance is large, the success probability is greater than that of a Poisson arrangement. These results characterize the performance of the system under geographical or MAC-induced clustering. We obtain the maximum intensity of transmitting nodes for a given outage constraint, i.e., the transmission capacity (of this spatial arrangement) and show that it is equal to that of a Poisson arrangement of nodes. For the analysis, techniques from stochastic geometry are used, in particular the probability generating functional of Poisson cluster processes, the Palm characterization of Poisson cluster processes, and the Campbell-Mecke theorem.

Journal ArticleDOI
TL;DR: In this article, the second-order coding rate of channel coding is discussed for general sequence of channels and the optimum secondorder transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method.
Abstract: In this paper, second-order coding rate of channel coding is discussed for general sequence of channels. The optimum second-order transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method. We apply this result to the discrete memoryless case, the discrete memoryless case with a cost constraint, the additive Markovian case, and the Gaussian channel case with an energy constraint. We also clarify that the Gallager bound does not give the optimum evaluation in the second-order coding rate.

Journal ArticleDOI
TL;DR: It is shown that a myopic policy that maximizes the immediate one-step reward is optimal when the state transitions are positively correlated over time and when the number of channels is limited to two or three, while presenting a counterexample for the case of four channels.
Abstract: This paper considers opportunistic communication over multiple channels where the state (ldquogoodrdquo or ldquobadrdquo) of each channel evolves as independent and identically distributed (i.i.d.) Markov processes. A user, with limited channel sensing capability, chooses one channel to sense and decides whether to use the channel (based on the sensing result) in each time slot. A reward is obtained whenever the user senses and accesses a ldquogoodrdquo channel. The objective is to design a channel selection policy that maximizes the expected total (discounted or average) reward accrued over a finite or infinite horizon. This problem can be cast as a partially observed Markov decision process (POMDP) or a restless multiarmed bandit process, to which optimal solutions are often intractable. This paper shows that a myopic policy that maximizes the immediate one-step reward is optimal when the state transitions are positively correlated over time. When the state transitions are negatively correlated, we show that the same policy is optimal when the number of channels is limited to two or three, while presenting a counterexample for the case of four channels. This result finds applications in opportunistic transmission scheduling in a fading environment, cognitive radio networks for spectrum overlay, and resource-constrained jamming and antijamming.

Journal ArticleDOI
TL;DR: Upper and lower bounds are derived on the capacity of the free-space optical intensity channel, which has a nonnegative input (representing the transmitted optical intensity), which is corrupted by additive white Gaussian noise.
Abstract: Upper and lower bounds are derived on the capacity of the free-space optical intensity channel. This channel has a nonnegative input (representing the transmitted optical intensity), which is corrupted by additive white Gaussian noise. To preserve the battery and for safety reasons, the input is constrained in both its average and its peak power. For a fixed ratio of the allowed average power to the allowed peak power, the difference between the upper and the lower bound tends to zero as the average power tends to infinity and their ratio tends to one as the average power tends to zero. When only an average power constraint is imposed on the input, the difference between the bounds tends to zero as the allowed average power tends to infinity, and their ratio tends to a constant as the allowed average power tends to zero.

Journal ArticleDOI
TL;DR: Simulation shows that network beamforming achieves the maximal diversity order and outperforms other existing schemes.
Abstract: This paper deals with beamforming in wireless relay networks with perfect channel information at the relays, receiver, and transmitter if there is a direct link between the transmitter and receiver. It is assumed that every node in the network has its own power constraint. A two-step amplify-and-forward protocol is used, in which the transmitter and relays not only use match filters to form a beam at the receiver but also adaptively adjust their transmit powers according to the channel strength information. For networks with no direct link, an algorithm is proposed to analytically find the exact solution with linear (in network size) complexity. It is shown that the transmitter should always use its maximal power while the optimal power of a relay ca.n take any value between zero and its maxima. Also, this value depends on the quality of all other channels in addition to the relay's own. Despite this coupling fact, distributive strategies are proposed in which, with the aid of a low-rate receiver broadcast, a relay needs only its own channel information to implement the optimal power control. Then, beamforming in networks with a direct link is considered. When the direct link exists during the first step only, the optimal power control is the same as that of networks with no direct link. For networks with a direct link during the second step only and both steps, recursive numerical algorithms are proposed. Simulation shows that network beamforming achieves the maximal diversity order and outperforms other existing schemes.

Journal ArticleDOI
TL;DR: A novel algorithm called SCALE is derived, that provides a significant performance improvement over the existing iterative water-filling (IWF) algorithm in multiuser DSL networks, doing so with comparable low complexity.
Abstract: Dynamic spectrum management of digital subscriber lines (DSLs) has the potential to dramatically increase the capacity of the aging last-mile copper access network. This paper takes an important step toward fulfilling this potential through power spectrum balancing. We derive a novel algorithm called SCALE, that provides a significant performance improvement over the existing iterative water-filling (IWF) algorithm in multiuser DSL networks, doing so with comparable low complexity. The algorithm is easily distributed through measurement and limited message passing with the use of a spectrum management center. We outline how overhead can be managed, and show that in the limit of zero message-passing, performance reduces to IWF.

Journal ArticleDOI
TL;DR: A fully quantum generalization of the classical asymptotic equipartition property is shown, where both the output of the experiment and side information are quantum.
Abstract: The classical asymptotic equipartition property is the statement that, in the limit of a large number of identical repetitions of a random experiment, the output sequence is virtually certain to come from the typical set, each member of which is almost equally likely. In this paper, a fully quantum generalization of this property is shown, where both the output of the experiment and side information are quantum. An explicit bound on the convergence is given, which is independent of the dimensionality of the side information. This naturally leads to a family of REacutenyi-like quantum conditional entropies, for which the von Neumann entropy emerges as a special case.

Journal ArticleDOI
TL;DR: It is shown that the speed of convergence of the k-NN method can be further improved by an adaptive choice of k.i.d., and the new universal estimator of divergence is proved to be asymptotically unbiased and mean-square consistent.
Abstract: A new universal estimator of divergence is presented for multidimensional continuous densities based on k-nearest-neighbor (k-NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k-NN method can be further improved by an adaptive choice of k.

Journal ArticleDOI
TL;DR: It is shown that, for this channel, Gaussian signalling in the form of beam-forming is optimal, and no pre-processing of information is necessary.
Abstract: We find the secrecy capacity of the 2-2-1 Gaussian MIMO wiretap channel, which consists of a transmitter and a receiver with two antennas each, and an eavesdropper with a single antenna. We determine the secrecy capacity of this channel by proposing an achievable scheme and then developing a tight upper bound that meets the proposed achievable secrecy rate. We show that, for this channel, Gaussian signalling in the form of beam-forming is optimal, and no pre-processing of information is necessary.

Journal ArticleDOI
TL;DR: A novel data representation scheme for multilevel flash memory cells, in which a set of n cells stores information in the permutation induced by the different charge levels of the individual cells, which eliminates the need for discrete cell levels when programming cells.
Abstract: We explore a novel data representation scheme for multilevel flash memory cells, in which a set of n cells stores information in the permutation induced by the different charge levels of the individual cells. The only allowed charge-placement mechanism is a ldquopush-to-the-toprdquo operation, which takes a single cell of the set and makes it the top-charged cell. The resulting scheme eliminates the need for discrete cell levels, as well as overshoot errors, when programming cells. We present unrestricted Gray codes spanning all possible n-cell states and using only "push-to-the-top" operations, and also construct balanced Gray codes. One important application of the Gray codes is the realization of logic multilevel cells, which is useful in conventional storage solutions. We also investigate rewriting schemes for random data modification. We present both an optimal scheme for the worst case rewrite performance and an approximation scheme for the average-case rewrite performance.

Journal ArticleDOI
TL;DR: This paper considers a more general signal model and assumes signals that live on or close to the union of linear subspaces of low dimension, and presents sampling theorems for this model that are in the same spirit as the Nyquist-Shannon sampling theorem in that they connect the number of required samples to certain model parameters.
Abstract: Compressed sensing is an emerging signal acquisition technique that enables signals to be sampled well below the Nyquist rate, given that the signal has a sparse representation in an orthonormal basis. In fact, sparsity in an orthonormal basis is only one possible signal model that allows for sampling strategies below the Nyquist rate. In this paper, we consider a more general signal model and assume signals that live on or close to the union of linear subspaces of low dimension. We present sampling theorems for this model that are in the same spirit as the Nyquist-Shannon sampling theorem in that they connect the number of required samples to certain model parameters. Contrary to the Nyquist-Shannon sampling theorem, which gives a necessary and sufficient condition for the number of required samples as well as a simple linear algorithm for signal reconstruction, the model studied here is more complex. We therefore concentrate on two aspects of the signal model, the existence of one to one maps to lower dimensional observation spaces and the smoothness of the inverse map. We show that almost all linear maps are one to one when the observation space is at least of the same dimension as the largest dimension of the convex hull of the union of any two subspaces in the model. However, we also show that in order for the inverse map to have certain smoothness properties such as a given finite Lipschitz constant, the required observation dimension necessarily depends logarithmically on the number of subspaces in the signal model. In other words, while unique linear sampling schemes require a small number of samples depending only on the dimension of the subspaces involved, in order to have stable sampling methods, the number of samples depends necessarily logarithmically on the number of subspaces in the model. These results are then applied to two examples, the standard compressed sensing signal model in which the signal has a sparse representation in an orthonormal basis and to a sparse signal model with additional tree structure.

Journal ArticleDOI
TL;DR: A general feedback channel coding theorem based on Massey's concept of directed information is proved and the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity.
Abstract: In this paper, we introduce a general framework for treating channels with memory and feedback. First, we prove a general feedback channel coding theorem based on Massey's concept of directed information. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. We give a recursive characterization of these sufficient statistics. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity. Fifth, scenarios with simple sufficient statistics are described. Sixth, error exponents for channels with feedback are presented.

Journal ArticleDOI
TL;DR: It is concluded that the techniques of relays, feedback, full-duplex operation and noisy cooperation do not increase the degrees of freedom of interference and X networks.
Abstract: We find the degrees of freedom of a network with S source nodes, R relay nodes, and D destination nodes, with random time-varying/frequency-selective channel coefficients and global channel knowledge at all nodes. We allow full-duplex operation at all nodes, as well as causal noise-free feedback of all received signals to all source and relay nodes. An outer bound to the capacity region of this network is obtained. Combining the outer bound with previous interference alignment based achievability results, we conclude that the techniques of relays, feedback, full-duplex operation and noisy cooperation do not increase the degrees of freedom of interference and X networks. As a second contribution, we show that for a network with K full-duplex nodes and K(K-1) independent messages with one message from every node to each of the other K-1 nodes, the total degrees of freedom are bounded above and below by [( K(K-1))/( (2K-2))] and [( K(K-1))/( (2K-3))], respectively.

Journal ArticleDOI
TL;DR: It is shown that D-TDMA is indeed optimal for achieving the ergodic sum capacity of the fading C-BC for all combinations of transmit-power and interference-power constraints.
Abstract: This paper studies the information-theoretic limits of a secondary or cognitive radio (CR) network under spectrum sharing with an existing primary radio network. In particular, the fading cognitive multiple-access channel (C-MAC) is first studied, where multiple secondary users transmit to the secondary base station (BS) under both individual transmit-power constraints and a set of interference-power constraints each applied at one of the primary receivers. This paper considers the long-term (LT) or the short-term (ST) transmit-power constraint over the fading states at each secondary transmitter, combined with the LT or ST interference-power constraint at each primary receiver. In each case, the optimal power allocation scheme is derived for the secondary users to achieve the ergodic sum capacity of the fading C-MAC, as well as the conditions for the optimality of the dynamic time-division multiple-access (D-TDMA) scheme in the secondary network. The fading cognitive broadcast channel (C-BC) that models the downlink transmission in the secondary network is then studied under the LT/ST transmit-power constraint at the secondary BS jointly with the LT/ST interference-power constraint at each of the primary receivers. It is shown that D-TDMA is indeed optimal for achieving the ergodic sum capacity of the fading C-BC for all combinations of transmit-power and interference-power constraints.

Journal ArticleDOI
TL;DR: This paper proposes a method to design error-correcting codes in the projective space by using a multilevel approach to design a new class of rank-metric codes and presents a decoding algorithm to the constructed codes.
Abstract: Coding in the projective space has received recently a lot of attention due to its application in network coding. Reduced row echelon form of the linear subspaces and Ferrers diagram can play a key role for solving coding problems in the projective space. In this paper, we propose a method to design error-correcting codes in the projective space. We use a multilevel approach to design our codes. First, we select a constant-weight code. Each codeword defines a skeleton of a basis for a subspace in reduced row echelon form. This skeleton contains a Ferrers diagram on which we design a rank-metric code. Each such rank-metric code is lifted to a constant-dimension code. The union of these codes is our final constant-dimension code. In particular, the codes constructed recently by Koetter and Kschischang are a subset of our codes. The rank-metric codes used for this construction form a new class of rank-metric codes. We present a decoding algorithm to the constructed codes in the projective space. The efficiency of the decoding depends on the efficiency of the decoding for the constant-weight codes and the rank-metric codes. Finally, we use puncturing on our final constant-dimension codes to obtain large codes in the projective space which are not constant-dimension.

Journal ArticleDOI
TL;DR: In this paper, the authors derived new achievable rates for the uplink channel of a cellular network with joint multicell processing (MCP), where unlike previous results, the ideal backhaul network has finite capacity per cell.
Abstract: In this work, new achievable rates are derived for the uplink channel of a cellular network with joint multicell processing (MCP), where unlike previous results, the ideal backhaul network has finite capacity per cell. Namely, the cell sites are linked to the central joint processor via lossless links with finite capacity. The new rates are based on compress-and-forward schemes combined with local decoding. Further, the cellular network is abstracted by symmetric models, which render analytical treatment plausible. For this family of idealistic models, achievable rates are presented for both Gaussian and fading channels. The rates are given in closed form for the classical Wyner model and the soft-handover model. These rates are then demonstrated to be rather close to the optimal unlimited backhaul joint processing rates, even for modest backhaul capacities, supporting the potential gain offered by the joint MCP approach. Particular attention is also given to the low-signal-to-noise ratio (SNR) characterization of these rates through which the effect of the limited backhaul network is explicitly revealed. In addition, the rate at which the backhaul capacity should scale in order to maintain the original high-SNR characterization of an unlimited backhaul capacity system is found.