1
Sum-Rate Maximization of NOMA Systems under
Imperfect Successive Interference Cancellation
Islam Abu Mahady, Ebrahim Bedeer, Salama Ikki, and Halim Yanikomeroglu
Abstract—This work addresses the sum-rate maximization for
a downlink non-orthogonal multiple access (NOMA) system in the
presence of imperfect successive interference cancellation (SIC).
We assume that the NOMA users adopt improper Gaussian
signalling (IGS), and hence, derive new expressions of their rates
under residual interference from imperfect SIC. We optimize
the circularity coefficient of the IGS-based NOMA system to
maximize its sum-rate subject to quality-of-service (QoS) require-
ments. Compared to the NOMA with proper Gaussian signaling
(PGS), simulation results show that the IGS-based NOMA system
demonstrates considerable sum-rate performance gain under
imperfect SIC.
Index Terms—Improper Gaussian signalling, non-linear opti-
mization, NOMA, sum-rate.
I. INTRODUCTION
Non-orthogonal multiple access (NOMA) proposes the
adoption of power/code domain to multiplex signal streams
from multiple users together and allow them to transmit
simultaneously using the same frequency/time/code resources
[1]. One of the advantages of NOMA systems is that when
the available resource blocks are assigned to weak channel
users, they can still be accessed by other strong channel users,
which qualifies NOMA techniques to achieve a higher overall
spectral efficiency (SE) [2], [3]. However, NOMA techniques
achieve this potential higher SE considering perfect successive
interference cancellation (SIC) (see, e.g., [1]–[4], and the
references therein). In real scenarios, the assumption of perfect
SIC at the receiver might not be practical, since there still
remain several serious implementation problems by using SIC,
e.g., error propagation and complexity scaling [1]. In [5], a
unified framework is presented assuming imperfect SIC, which
shows that the performance converges to an error floor at
the high signal-to-noise ratio (SNR) region and obtain a zero
diversity order. Hence it is of great interest to compensate the
impact of imperfect SIC for the NOMA systems.
Recent research works have shown that improper Gaussian
signalling (IGS) has the potential over proper (conventional)
Gaussian signalling (PGS) to enhance the overall achievable
rate of systems that suffer from interference [6]. Compared to
the PGS scheme which assumes independent real and imag-
inary signal components with equal power, the IGS scheme
loosens these constraints and introduce a circularity coefficient
that enables a more general Gaussian signaling scheme [7].
To the best of the authors’ knowledge, there is no existing
work in the literature that exploits IGS in an effort to maximize
the overall sum-rate of the NOMA system under the practical
I. Abu Mahady and S. Ikki are with Lakehead University, ON, Cananda.
Emails:{iabumah, sikki}@lakeheadu.ca. E. Bedeer is with Ulster University,
UK. Email: e.bedeer.mohamed@ulster.ac.uk, and H. Yanikomeroglu is with
Carleton University, ON, Canada. Email: halim@sce.carleton.ca.
assumption of imperfect SIC, which motivates us to develop
this work. In particular, new closed-form expressions for
the users’ rate are derived for a downlink two-user NOMA
system in the presence of imperfect SIC. Using the derived
expressions, an optimization problem is formulated to optimize
the circularity coefficient to maximize the overall NOMA
sum-rate subject to minimum rate requirements constraints.
Simulation results show a considerable sum-rate performance
gain when using IGS-based NOMA systems compared with
PGS-based NOMA systems.
II. PRELIMINARY: IMPROPER RANDOM VECTORS
In this section, preliminary of IGS definitions are presented
to ease the understanding of the derivation of information rates
of NOMA users.
A complex random variable (RV) is called proper if its
pseudo-variance is equal to zero, otherwise it is called im-
proper [8]. For a complex RV x
i
, we use C
x
i
and
ˆ
C
x
i
to
denote the covariance and pseudo-covariance, respectively.
Then for the zero-mean input Gaussian signal x
i
, ∀i, we have
C
x
i
= E[x
i
x
∗
i
],
ˆ
C
x
i
= E[x
i
x
i
], and the impropriety degree
of x
i
is given as
κ
x
i
= |
ˆ
C
x
i
|/C
x
i
, ∀i, (1)
where 0 ≤ κ
x
i
≤ 1. If κ
x
i
= 0, we say that x
i
is proper, and if
κ
x
i
= 1, we have maximally improper signal. Note that C
x
i
is nonnegative real number equal to the power value of the
transmitted signal, while
ˆ
C
x
i
is complex number in general.
III. SYSTEM MODEL AND RATE ANALYSIS
We consider a downlink NOMA system with two users
(strong channel user and weak channel user) and a base
station. The channel coefficient between the base station and
user i is denoted by h
i
, ∀i = 1, 2, that is modelled as a
complex Gaussian RV with zero-mean and variance σ
2
h
i
. The
noise at the receivers ends are modelled as zero-mean additive
white Gaussian random variable with variances σ
2
n
. Different
from the conventional setup where PGS is assumed, in this
work, user’s 1 signal x
1
and user’s 2 signal x
2
are zero-mean
complex Gaussian RVs which can be improper. Without loss
of generality, it is assumed that |h
1
|
2
> |h
2
|
2
, i.e. user 1 is
with strong channel gain and user 2 is with weak channel gain.
According to the NOMA principle, the transmit power of the
weak user’s signal must be greater than that of the strong user,
i.e., P
2
> P
1
. Hence, user 2 decodes directly its signal because
the interference inflicted by the user 1 is small and can thus be
treated as noise. In contrast, user 1 can decode its own signal
after cancelling the weak user’s decoded signal through a SIC
detector [1]. We assume that SIC process at user’s 1 receiver
2
is imperfect and the residual interference component due to
this imperfection is quantified by a factor β (0 ≤ β ≤ 1),
where β = 0 referes to perfect SIC and β = 1 refers to the
fully imperfect SIC.
The input-output relationship for the two-user SISO system
can be expressed as
y
1
=
p
P
1
h
1
s
1
+ β
p
P
2
h
1
s
2
+ n
1
, (2)
y
2
=
p
P
2
h
2
s
2
+
p
P
1
h
2
s
1
+ n
2
, (3)
where s
i
is ith signal and n
i
is AWGN at the corresponding
receivers.
In the following, we derive the rate expressions for the gen-
eral case of IGS for for both users, i.e., x
1
and x
2
are improper.
Let x
i
=
√
P
i
s
i
, ∀i = 1, 2 are the independent signals for user
1 and 2, respectively, and denote the covariance and pseudo-
covariance of the transmit signal by
C
x
i
= P
i
C
s
i
, (4)
ˆ
C
x
i
= P
i
ˆ
C
s
i
, ∀i = 1, 2, (5)
where C
s
i
= E[s
i
s
∗
i
] and
ˆ
C
s
i
= E[s
i
s
i
]. We assume that
C
s
i
= E[s
i
s
∗
i
] = 1, ∀i = 1, 2, i.e., transmit a symbol with a
unit power. Next, we derive the rate expressions in terms of
circularity coefficient. The covariance and pseudo-covariance
of y
i
, i = 1, 2, can be obtained from (2) and (3) as
C
y
1
= P
1
|h
1
|
2
+ β
2
P
2
|h
1
|
2
+ σ
2
1
, (6)
ˆ
C
y
1
= P
1
κ
s
1
h
2
1
+ β
2
P
2
κ
s
2
h
2
1
, (7)
C
y
2
= P
2
|h
2
|
2
+ P
1
|h
2
|
2
+ σ
2
2
, (8)
ˆ
C
y
2
= P
2
κ
s
2
h
2
2
+ P
1
κ
s
1
h
2
2
. (9)
Define the noise and the interference-plus-noise terms in (2)
and (3), as z
i
, i = 1, 2, at each receiver, respectively, where
z
1
= n
1
and z
2
=
√
P
1
h
2
s
1
+ n
2
, we get
C
z
1
= σ
2
1
,
ˆ
C
z
1
= 0, C
z
2
= P
1
|h
2
|
2
+ σ
2
2
, and
ˆ
C
z
2
= P
1
κ
s
1
h
2
2
.
(10)
Following [8], the achievable rate expression for a two-user
SISO system is given as [6]
R
i
=
1
2
log
2
C
2
y
i
− |
ˆ
C
y
i
|
2
C
2
z
i
− |
ˆ
C
z
i
|
2
!
. (11)
By substituting (6), (7), and (10) into (11), and assuming
without loss of generality σ
2
1
= σ
2
2
= σ
2
, the achievable rate
expression for the strong user 1 of a NOMA system, in the
case of both users adopt IGS, reduces to
R
1
(κ
s
1
, κ
s
2
) = log
2
1 +
P
1
|h
1
|
2
β
2
P
2
|h
1
|
2
+ σ
2
| {z }
proper
+
1
2
log
2
1 −
|h
2
1
P
1
κ
s
1
|
2
+ |β
2
P
2
2
|h
1
|
2
κ
s
2
|
2
(P
1
|h
1
|
2
+ β
2
P
2
|h
1
|
2
+ σ
2
)
2
| {z }
Improper
−
1
2
log
2
1 −
|β
2
P
2
2
|h
1
|
2
κ
s
2
|
2
(β
2
P
2
|h
1
|
2
+ σ
2
)
2
| {z }
Improper
. (12)
Similarly, by substituting (8), (9), and (10) into (11), the
achievable rate expression for the weak user 2 of a NOMA
system, in the case of both users use IGS, reduces to
R
2
(κ
s
1
, κ
s
2
) = log
2
1 +
P
2
|h
2
|
2
P
1
|h
2
|
2
+ σ
2
| {z }
Proper
+
1
2
log
2
1 −
|P
2
h
2
2
κ
s
2
|
2
+ |P
1
h
2
2
κ
s
1
|
2
(P
2
|h
2
|
2
+ P
1
|h
2
|
2
+ σ
2
)
2
| {z }
Improper
−
1
2
log
2
1 −
|P
1
h
2
2
κ
s
1
|
2
(P
1
|h
2
|
2
+ σ
2
)
2
| {z }
Improper
. (13)
Please note that each R
i
, i = 1, 2, in (12), (13) includes two
parts; proper and improper. Substituting κ
s
i
= 0, ∀i = 1, 2
and β = 0 reduces to the rates of PGS case in perfect SIC,
which proves the correctness of the derived expressions.
IV. OPTIMIZATION PROBLEM
In this section, an optimization problem is formulated to
optimize the IGS circularity coefficient in order to maximize
the sum-rate of a two-user SISO NOMA system subject
to minimum rate requirements of each user. Due to space
limitations, we focus on the case where we use IGS for
strong user (i.e., x
1
is improper and κ
s
1
6= 0) and PGS
for weak user (i.e., x
2
is proper and κ
s
2
= 0). Other cases
will be investigated in future work. Also, we assume the
powers P
1
and P
2
are already allocated to user 1 and 2,
respectively (i.e., they are not optimization variables). That
said, the optimization problem for maximizing the sum-rate
under QoS constraints can be formulated as
maximize
κ
s
1
R
1
(κ
s
1
) + R
2
(κ
s
1
)
subject to C1 : R
1
(κ
s
1
) ≥ R
min
1
,
C2 : R
2
(κ
s
1
) ≥ R
min
2
,
C3 : 0 ≤ κ
s
1
≤ 1,
(14)
where R
1
(κ
s
1
) and R
2
(κ
s
1
) are obtained from (12) and (13),
respectively, at κ
s
2
= 0. R
min
1
and R
min
2
are the minimum
rate requirements of the strong user and the weak user,
respectively. The constraint C3 reflects that the circulatory
coefficient is between 0 and 1 as shown in Definition 2.
The optimization problem in (14) can be solved by applying
the Karush-Kuhn-Tucker (KKT) conditions; however, it is
worthy to mention that the obtained circularity coefficient κ
∗
s
1
will be sub-optimal as the the problem in (14) is non-convex.
The Lagrangian function can be expressed as
L(κ
s
1
, λ
1
, λ
2
) = −(R
1
(κ
s
1
) + R
2
(κ
s
1
))
+λ
1
(R
min
1
− R
1
(κ
s
1
)) + λ
2
(R
min
2
− R
2
(κ
s
1
)), (15)
where λ
1
and λ
2
are the non-negative Lagrange multipliers
associated with the QoS constraints of user 1 and 2, respec-
tively. The circularity coefficient constraint not considered in
3
κ
2
s
1
= 0.5
(Φ + Ψ) + (Φ − Ψ)
1 + λ
2
1 + λ
1
−0.5
"
(Φ − Ψ)
2
1 +
1 + λ
2
1 + λ
1
2
!
+ (Φ − Ψ)
1 + λ
2
1 + λ
1
(2(Φ + Ψ) − 4Ω)
#!
1
2
. (23)
the Lagrangian function will be satisfied later. That said, the
KKT conditions can be written as follows [9]
∂L(κ
∗
s
1
, λ
1
, λ
2
)
∂κ
s
1
= 0, (16)
λ
1
(R
min
1
− R
1
(κ
∗
s
1
)) = 0, (17)
λ
2
(R
min
2
− R
2
(κ
∗
s
1
)) = 0, (18)
R
min
1
− R
1
(κ
∗
s
1
) ≤ 0, (19)
R
min
2
− R
2
(κ
s
∗
1
) ≤ 0, (20)
λ
1
, λ
2
≥ 0. (21)
From (16), we can obtain the circularity coefficient κ
∗
s
1
as in
(23) on top of this page, where Φ =
1 +
σ
2
P
1
|h
2
|
2
2
, Ψ =
1 +
P
2
P
1
+
σ
2
P
1
|h
2
|
2
2
, and Ω =
1 + β
2
P
2
P
1
+
σ
2
P
1
|h
1
|
2
2
. To
consider C3, we need to guarantee that the term under the
square root in (23) is positive and also the first term of (23)
is greater than the second term of it. The values of λ
1
and
λ
2
in (23) can be computed using the subgradient method [9]
as follows.
λ
l+1
i
=
λ
l
i
− α
l
i
(R
i
− R
min
i
)
+
, ∀i = 1, 2, (24)
where [.]
+
is defined as max(., 0) and α
i
is a sufficiently small
step size chosen to equals 0.1/
√
l where l is the iteration
number [9]. However, one can notice from (17) that either
λ
1
= 0 or R
1
(κ
∗
s
1
) = R
min
1
. Similarly, (18) implies that
either λ
2
= 0 or R
2
(κ
∗
s
1
) = R
min
2
. That said, four possible
cases exist, as follows.
– Case 1: λ
1
= 0 and λ
2
= 0 means that both QoS
constraints of user 1 and user 2 are inactive.
– Case 2: λ
1
= 0 and λ
2
6= 0 implies that the sub-optimal
circularity coefficient exists when R
2
(κ
∗
s
1
) = R
min
2
.
– Case 3: λ
1
6= 0 and λ
2
= 0 implies that the sub-optimal
circularity coefficient exists when R
1
(κ
∗
s
1
) = R
min
1
.
– Case 4: λ
1
6= 0 and λ
2
6= 0 implies that if the problem
is feasible, the sub-optimal circularity coefficient exists when
both R
1
(κ
∗
s
1
) = R
min
1
and R
2
(κ
∗
s
1
) = R
min
2
.
The proposed algorithm to solve the problem in (14) can be
formally summarized as follows.
1) Input: R
min
1
, R
min
2
, P
1
, P
2
, h
1
, h
2
, σ
2
, and β.
2) Set λ
1
= λ
2
= 0. Calculate κ
s
1
from (23). Calculate R
1
and R
2
from (12) and (13), respectively.
3) if R
1
≥ R
min
1
and R
2
≥ R
min
2
, then, the sub-optimal
solution κ
∗
s
1
is reached.
4) else if R
1
< R
min
1
and R
2
≥ R
min
1
, then, find non-
negative λ
1
from (24) such that R
1
(κ
s
1
) = R
min
1
and
re-calculate κ
s
∗
1
from (23). Repeat until convergence.
5) else if R
1
≥ R
min
1
and R
2
< R
min
1
, then, find non-
negative λ
2
from (24) such that R
2
(κ
s
1
) = R
min
2
and
re-calculate κ
s
∗
1
from (23). Repeat until convergence.
6) else R
1
< R
min
1
and R
2
< R
min
1
, then, find
non-negative λ
1
and λ
2
from (24) if exists such that
R
1
(κ
s
1
) = R
min
1
and R
2
(κ
s
1
) = R
min
2
and re-
calculate κ
s
∗
1
from (23). Repeat until convergence.
7) Output: κ
s
∗
1
.
V. SIMULATION RESULTS
In this section, we simulated a downlink two-user served by
a base station in a NOMA system employing IGS and compare
its achieved sum rate (R
1
+ R
2
) to its counterpart of PGS-
based NOMA systems. Unless otherwise mentioned, R
min
1
=
R
min
2
= 1.2 bits/sec/Hz, P
1
= 0.3P
T
, and P
2
= 0.7P
T
.
In Fig. 1, the sum-rate is simulated versus SNR =
P
T
σ
2
n
,
where P
T
is the total transmit power, at different values of
β. As can be seen, the IGS-based NOMA system outperforms
PGS-based NOMA for all levels of imperfect SIC. In particu-
lar, as the SIC becomes worse, i.e., β = 0.4, the sum-rate gain
of using IGS increases over PGS NOMA. IGS also offers a
good gain in the low SNR region as the effect of the imperfect
SIC is significant on the users’ rate. At high SNR, the PGS-
based NOMA system approaches the sum-rate performance of
the IGS-based NOMA system. In addition, we use exhaustive
search method to find the optimal solution and compare it with
the proposed KKT sub-optimal solution. The results show that
there is a small performance gap between the optimal solution
and proposed sup-optimal solution in terms of sum-rate at low
SNR values and the gap tends to zero at high SNR values. It is
worthy note that the proposed solution is far less complex than
the optimal solution of the exhaustive search. The figure also
shows that in case of perfect SIC, i.e., β = 0, both schemes
perform similarly.
In Fig. 2, the sum-rate vs SNR for different values of P
1
, P
2
at β = 0.3 is simulated for both IGS and PGS NOMA system.
As the strong user gains more power, i.e., P
1
becomes larger,
the sum-rate curves shift up and show higher sum-rate. One
can notice from Fig. 2 that different power allocation ratios do
not affect the gain of IGS over PGS based NOMA systems.
The effect of users’ channel strength on the sum-rate for
IGS-based NOMA performance is shown in Fig. 3. The sum-
rate is simulated for the case of σ
2
h
1
= {1, 2, 3, 5, 7, 9}σ
2
h
2
at
P
1
= 0.1P
T
, P
2
= 0.9P
T
, and β = 0.3. It is clear that as σ
2
h
1
increases, the sum-rate enhances, i.e., as the channel of the first
user becomes stronger, its rate becomes higher. Meanwhile,
the rate of the user with weak channel is maximized by the
proposed approach through the IGS-NOMA concept.
In Fig. 4, we show the convergence of the proposed al-
gorithm at different values of β. On average, the algorithm
needs small number of iterations to converge and the number
increases as β increases.
4
0 2 4 6 8 10 12 14 16 18 20
SNR (dB)
2.5
3
3.5
4
4.5
5
5.5
Sum-rate (bits/sec/Hz)
IGS-NOMA (sub-optimal - KKT sol)
PGS-NOMA
IGS-NOMA (optimal - exhaustive search)
0 5 10 15 20
2
4
6
8
10
12
PGS-NOMA, = 0
IGS-NOMA, = 0
Fig. 1: Sum-rate vs SNR for IGS-based and PGS-based NOMA
systems for different β.
0 2 4 6 8 10 12 14 16 18 20
SNR (dB)
2
2.5
3
3.5
4
4.5
Sum-rate (bits/sec/Hz)
Fig. 2: Sum-rate vs SNR for IGS-based and PGS-based NOMA
systems for different P
1
, P
2
values, with β = 0.3.
VI. CONCLUSION
In this work, we optimized the sum-rate of a two-user
NOMA system subject to minimum QoS requirements under
imperfect SIC. An iterative algorithm was developed to find
the sub-optimal IGS circularity coefficient that maximizes the
sum-rate. Simulation results showed that IGS improves the
sum-rate at low-to-medium SNR region. In addition, it was
observed that the gain from IGS increases when imperfect
SIC gets higher. Results also revealed that the power allocation
does not affect the gain of IGS-based NOMA systems over its
PGS-based counterpart. Moreover, sum-rate increases when
channel gain ratio between users increases, but this improve-
ment saturates at high SNR values.
REFERENCES
[1] M. Zeng, A. Yadav, O. A. Dobre, G. I. Tsiropoulos, and H. V. Poor,
“Capacity comparison between MIMO-NOMA and MIMO-OMA with
multiple users in a cluster,” IEEE Journal Sel. Areas in Commun., vol.
35, no. 10, pp. 2413–2424, Oct. 2017.
[2] Y. Liu, Z. Qin, M. Elkashlan, A. Nallanathan, and J. A. McCann, “Non-
orthogonal multiple access in large-scale heterogeneous networks,” IEEE
Journal Sel. Areas in Commun., vol. 35, no. 12, pp. 2667–2680, Dec.
2017.
0 2 4 6 8 10 12 14 16 18 20
SNR (dB)
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Sum-rate (bits/sec/Hz)
IGS-NOMA
Fig. 3: Sum-rate vs SNR for IGS-based NOMA system for different
σ
2
h
1
to σ
2
h
2
ratios, with β = 0.3, P
1
= 0.1P
T
, and P
2
= 0.9P
T
.
0 10 20 30 40 50 60 70 80 90 100
Number of iterations
0
1
2
3
4
5
6
Sum Rate (bits/s/Hz)
Fig. 4: Sum-rate vs number of iterations for algorithm convergence
with β = 0.2, 0.3, 0.4.
[3] L. P. Qian, Y. Wu, H. Zhou, and X. Shen, “Joint uplink base station asso-
ciation and power control for small-cell networks with non-orthogonal
multiple access,” IEEE Trans. Wireless Commun., vol. 16, no. 9, pp.
5567–5582, Sept. 2017.
[4] Y. Sun, D. W. K. Ng, Z. Ding, and R. Schober, “Optimal joint power and
subcarrier allocation for full-duplex multicarrier non-orthogonal multiple
access systems,” IEEE Trans. Commun., vol. 65, no. 99, pp. 1077–1091,
Mar. 2017.
[5] X. Yue, Z. Qin, Y. Liu, S. Kang, and Y. Chen, “A unified framework
for non-orthogonal multiple access,” to appear, IEEE Trans. Commun.,
vol. 66, no. 11, pp. 5346–5359, Nov. 2018.
[6] Y. Zeng, C. M. Yetis, E. Gunawan, Y. L. Guan, and R. Zhang, “Transmit
optimization with improper Gaussian signaling for interference chan-
nels,” IEEE Trans. Sig. Process., vol. 61, no. 11, pp. 2899–2913, Jun.
2013.
[7] O. Amin, W. Abediseid, and M. Alouini, “Overlay spectrum sharing
using improper Gaussian signaling,” IEEE Journal Sel. Areas in
Commun., vol. 35, no. 1, pp. 50–62, Jan. 2017.
[8] F. D. Neeser and J. L. Massey, “Proper complex random processes with
applications to information theory,” IEEE Trans. Inf. Theory, vol. 39,
no. 4, pp. 1293–1302, Jul. 1993.
[9] S. Boyd, L. Xiao, and A. Mutapcic, “Subgradient methods,” Lecture
Notes of EE392o, Stanford University, Autumn Quarter, 2003.
