APPEARED IN IEEE SP MAGAZINE, SPECIAL ISSUE ON “CONVEX OPT. FOR SP”, MAY 2010 1
Semidefinite Relaxation of
Quadratic Optimization Problems
Zhi-Quan Luo, Wing-Kin Ma, Anthony Man-Cho So, Yinyu Ye, and Shuzhong Zhang
I. INTRODUCTION
In recent years, the semidefinite relaxation (SDR) technique
has been at the center of some of the very exciting develop-
ments in the area of signal processing and communications,
and it has shown great significance and relevance on a va-
riety of applications. Roughly speaking, SDR is a powerful,
computationally efficient approximation technique for a host
of very difficult optimization problems. In particular, it can be
applied to many nonconvex quadratically constrained quadratic
programs (QCQPs) in an almost mechanical fashion. These
include the following problems:
min
x∈R
n
x
T
Cx
s.t. x
T
F
i
x ≥ g
i
, i = 1, . . . , p,
x
T
H
i
x = l
i
, i = 1, . . . , q,
(1)
where the given matrices C, F
1
, . . . , F
p
, H
1
, . . . , H
q
are as-
sumed to be general real symmetric matrices, possibly in-
definite. The class of nonconvex QCQPs (1) captures many
problems that are of interest to the signal processing and com-
munications community. For instance, consider the Boolean
quadratic program (BQP)
min
x∈R
n
x
T
Cx
s.t. x
2
i
= 1, i = 1, . . . , n.
(2)
The BQP is long-known to be a computationally difficult
problem. In particular, it belongs to the class of NP-hard
problems. Nevertheless, being able to handle the BQP well
has an enormous impact on multiple-input-multiple-output
(MIMO) detection and multiuser detection. Another important
yet NP-hard problem in the nonconvex QCQP class (1) is
min
x∈R
n
x
T
Cx
s.t. x
T
F
i
x ≥ 1, i = 1, . . . , m ,
(3)
where C, F
1
, . . . , F
m
are all positive semidefinite. Prob-
lem (3) captures the multicast downlink transmit beamforming
problem; see [1] for details. An illustration of an instance of
Problem (3) is provided in Fig. 1. As seen from the figure,
the feasible set of (3) is the intersection of the exteriors of
multiple ellipsoids, which makes the problem difficult.
As a matter of fact, SDR has been studied and applied in
the optimization community long before it made its impact
on signal processing and communications. The idea of SDR
can already be found in an early paper of Lov´asz in 1979
[2], but it was arguably the seminal work of Goemans and
Williamson in 1995 [3] that sparked the significant interest
in and rapid development of SDR techniques. In that work, it
was shown that SDR can be used to provide an approximation
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
x
1
x
2
Fig. 1. A nonconvex QCQP in R
2
. Colored lines: contour of the objective
function; gray area: the feasible set; black lines: boundary of each constraint.
accuracy of no worse than 0.8756 for the Maximum Cut
problem (the BQP with some conditions on C). In other
words, even though the Maximum Cut problem is NP-hard,
one could efficiently obtain a solution whose objective value
is at least 0.8756 times the optimal value using SDR. Since
then, we have seen a number of dedicated theoretical analyses
that establish the SDR approximation accuracy under different
problem settings [3]–[11], and that have greatly improved our
understanding of the capabilities of SDR. Today, we are even
able to pin down a number of conditions under which SDR
provides an exact optimal solution to the original problem [7],
[12]–[16].
In the field of signal processing and communications, the in-
troduction of SDR since the early 2000’s has reshaped the way
we see many topics today. Many practical experiences have
already indicated that SDR is capable of providing accurate
(and sometimes near-optimal) approximations. For instance,
in MIMO detection, SDR is now known as an efficient high-
performance approach [17]–[23] (see also [24]–[26] for blind
MIMO detection). The promising empirical approximation
performance of SDR has motivated new endeavors, leading
to the creation of new research trends in some cases. One
such example is in the area of transmit beamforming, which
has attracted much recent interest; for a review of this exciting
topic, please see the article by Gershman et al. in this special
issue [1], and [27]. The effectiveness of transmit beamforming
depends much on how well one can handle (often nonconvex)
QCQPs, and its technical progress could have been slower
if SDR had not been known to the signal processing com-
munity. Another example worth mentioning is sensor network
2 APPEARED IN IEEE SP MAGAZINE, SPECIAL ISSUE ON “CONVEX OPT. FOR SP”, MAY 2010
localization, a practically important but technically challenging
problem. SDR has proven to be an effective technique for
tackling the sensor network localization problem, both in
theory and practice [28]–[31]. In addition to the three major
applications mentioned above, there are many other different
applications of SDR, such as waveform design in radar [32],
[33], phase unwrapping [34], robust blind beamforming [35],
large-margin parameter estimation in speech recognition (see
the article by Jiang and Li in this special issue [36] for further
details), transmit B
1
shim in MRI [37], and many more [38]–
[41]. It is anticipated that SDR would find more applications
in the near future.
This paper aims to give an overview of SDR, with an
emphasis on showing the underlying intuitions and various
applications of this powerful tool. In fact, we will soon see
that the implementation of SDR can be very easy, and that
allows signal processing practitioners to quickly test the via-
bility of SDR in their applications. Several highly successful
applications will be showcased as examples. We will also
endeavor to touch on some advanced, key theoretical results
by highlighting their practical impacts and implications.
This paper is organized as follows. Section II describes the
basic ideas of SDR and its operations. Section III showcases
an SDR application, namely, MIMO detection. In Section IV
we shed light into the randomization concept, which plays an
indispensable role in both theoretical and practical advances
of SDR. Section V considers extensions of SDR to more
general cases. This is immediately followed by Section VI,
where another application example, B1-shimming in MRI,
is demonstrated. Section VII presents a theoretical subject,
namely SDR rank reduction, which has important implications
for the tightness of SDR approximation. Section VIII describes
the application of SDR in sensor network localization. We
draw conclusions and discuss further issues in Section IX.
II. THE CONCEPT OF SEMIDEFINITE RELAXATION
To make the notation more concise, let us write our problem
of interest—namely, the real-valued homogeneous QCQP in
(1)—as follows:
min
x∈R
n
x
T
Cx
s.t. x
T
A
i
x D
i
b
i
, i = 1, . . . , m.
(4)
Here, ‘D
i
’ can represent either ‘≥’, ‘=’, or ‘≤’ for each i;
and C, A
1
, . . . , A
m
∈ S
n
, where S
n
denotes the set of all
real symmetric n ×n matrices; and b
1
, . . . , b
m
∈ R. A crucial
first step in deriving an SDR of Problem (4) is to observe that
x
T
Cx = Tr(x
T
Cx) = Tr(Cxx
T
),
x
T
A
i
x = Tr(x
T
A
i
x) = Tr(A
i
xx
T
).
In particular, both the objective function and constraints in
(4) are linear in the matrix xx
T
. Thus, by introducing a new
variable X = xx
T
and noting that X = x x
T
is equivalent to
X being a rank one symmetric positive semidefinite (PSD)
matrix, we obtain the following equivalent formulation of
Problem (4):
min
X∈S
n
Tr(CX)
s.t. Tr(A
i
X) D
i
b
i
, i = 1, . . . , m,
X 0, rank(X) = 1.
(5)
Here, we use X 0 to indicate that X is PSD.
At this point, it may seem that we have not achieved much,
as Problem (5) is just as difficult to solve as Problem (4).
However, the formulation in (5) allows us to identify the
fundamental difficulty in solving Problem (4). Indeed, the only
difficult constraint in (5) is the rank constraint rank(X) = 1,
which is nonconvex (the objective function and all other
constraints are convex in X). Thus, we may as well drop
it to obtain the following relaxed version of Problem (4):
min
X∈S
n
Tr(CX)
s.t. Tr(A
i
X) D
i
b
i
, i = 1, . . . , m,
X 0.
(6)
Problem (6) is known as an SDR of Problem (4), where the
name stems from the fact that (6) is an instance of semidefinite
programming (SDP). The upshot of the formulation in (6) is
that it can be solved, to any arbitrary accuracy, in a numerically
reliable and efficient fashion. In fact, SDRs can now be
handled very conveniently and effectively by readily available
(and free) software packages. Let us give an example: Suppose
that ‘D
i
’ equal ‘≥’ for i = 1, . . . , p, and ‘D
i
’ equal ‘=’
for i = p + 1, . . . , m. Using the convex optimization toolbox
CVX [42], we can solve (6) in MATLAB with the following
piece of code:
Box 1. A CVX code for SDR
cvx_begin
variable X(n,n) symmetric
minimize(trace(C
*
X));
subject to
for i=1:p
trace(A(:,:,i)
*
X) >= b(i);
end
for i=p+1:m
trace(A(:,:,i)
*
X) == b(i);
end
X == semidefinite(n);
cvx_end
While advances in convex optimization and software have
enabled us to solve SDPs easily and transparently, one might
question how effective is the process (how fast or slow it would
be?). In the backstage most convex optimization toolboxes
handle SDPs using an interior-point algorithm, a sophisticated
topic in its own right (see, e.g., [43]). Simply speaking, the
SDR problem (6) can be solved with a worst case complexity
of
O(max{m, n}
4
n
1/2
log(1/ ))
given a solution accuracy > 0
1
. The complexity above does
not assume sparsity or any special structures in the data matri-
ces C, A
1
, . . . , A
m
. Some algorithms, such as SeDuMi [46]
1
Our reported complexity order is obtained by counting the arithmetic
operations of a specific interior-point method, namely the primal-dual path-
following method in [44]. See [45] for a more detailed description on the
operation count.
LUO, MA, SO, YE, & ZHANG, SDR OF QUADRATIC OPT. PROBLEMS 3
(employed as one of the core solvers in CVX), can utilize
data matrix sparsity to speed up the solution process. We also
refer the readers to the article [47] in this special issue for
other fast real-time convex optimization solvers. For certain
specially structured SDR problems, one can even exploit
the problem structures to build fast customized interior-point
algorithms. For example, for BQP, a custom-built interior-
point algorithm [44] can solve SDR with a complexity of
O(n
3.5
log(1/)) [instead of O(n
4.5
log(1/ ))]. Furthermore,
the SDR complexity scales slowly (logarithmically) with and
most applications do not require a very high solution precision.
Hence, simply speaking, we can say that
SDR is a computationally efficient approximation approach
to QCQP, in the sense that its complexity is polynomial in
the problem size n and the number of constraints m.
Of course, there is no free lunch in turning the NP-hard
Problem (4) (which is equivalent to Problem (5)) into the
polynomial-time solvable Problem (6). Indeed, a fundamental
issue that one must address when using SDR is how to convert
a globally optimal solution X
?
to Problem (6) into a feasible
solution
˜
x to Problem (4). Now, if X
?
is of rank one, then
there is nothing to do, for we can write X
?
= x
?
x
?
T
,
and x
?
will be a feasible—and in fact optimal—solution
to Problem (4). On the other hand, if the rank of X
?
is
larger than 1, then we must somehow extract from it, in an
efficient manner, a vector
˜
x that is feasible for Problem (4).
There are many ways to do this, and they generally follow
some intuitively reasonable heuristics (true even in engineering
sense). However, we must emphasize that even though the
extracted solution is feasible for Problem (4), it is in general
not an optimal solution (for otherwise we would have solved
an NP-hard problem in polynomial time).
As an illustration, consider the intuitively appealing idea of
applying a rank-one approximation on X
?
. Specifically, let
r = rank(X
?
), and let
X
?
=
r
X
i=1
λ
i
q
i
q
T
i
denote the eigen-decomposition of X
?
, where λ
1
≥ λ
2
≥
. . . ≥ λ
r
> 0 are the eigenvalues and q
1
, . . . , q
r
∈ R
n
are the
respective eigenvectors. Since the best rank-one approximation
X
?
1
to X
?
(in the least 2-norm sense) is given by X
?
1
=
λ
1
q
1
q
T
1
, we may define
˜
x =
√
λ
1
q
1
as our candidate solution
to Problem (4), provided that it is feasible. Otherwise, we can
try to map
˜
x to a “nearby” feasible solution
ˆ
x of Problem (4).
In general, such a mapping is problem dependent, but it can be
quite simple. For example, for the BQP (2) where x
2
i
= 1 for
all i, we can obtain a feasible solution from
˜
x via
ˆ
x = sgn(
˜
x),
where sgn(·) is the element-wise signum function.
Our basic description of SDR is now complete. Before we
proceed, some remarks are in order.
1) Now that we have seen one method of extracting a
feasible solution
ˆ
x to Problem (4) from a solution
X
?
to the SDP (6), it is natural to ask what is the
quality of the extracted solution
ˆ
x. It turns out that
there are several measures available to address this issue.
Although we will not discuss them at this point, it should
be emphasized that regardless of which measure we use,
the quality will certainly depend on the method by which
we extract the solution
ˆ
x.
2) Apart from the rank relaxation interpretation of SDR
as described above, there is another interpretation that
is based on Lagrangian duality. Specifically, it can be
shown that the SDR (6) is a Lagrangian bidual of the
original problem (4). We refer the reader to, e.g., [48]
for details.
III. APPLICATION: MIMO DETECTION
Let us show an example of SDR application before pro-
ceeding to further advanced concepts and applications.
The problem we consider is MIMO detection, a frequently
encountered problem in digital communications. To put it into
context, consider a generic N -input M-output model
y
C
= H
C
s
C
+ v
C
. (7)
Here, y
C
∈ C
M
is the received vector, H
C
∈ C
M×N
is
the MIMO channel, s
C
∈ C
N
is the transmitted symbol
vector, and v
C
∈ C
M
is an additive white Gaussian noise
vector. Eq. (7) is popularly used to model point-to-point
multiple-antenna systems such as the spatial multiplexing (or
V-BLAST) depicted in Fig. 2. In fact, it is known (see, e.g.,
[49]) that the same model as in (7) can be used to formulate
detection problems in many other communication scenarios,
such as multiuser systems, space-time coding systems, space-
frequency coding systems, and combinations such as multiuser
multi-antenna systems. The wide applicability of the MIMO
model (7) makes its respective detection problem attractive
and important to tackle.
Spatial
Multiplexer
. . . . . .
. . . . . .
MIMO
Detector
Symbols
s
C
Detected
Symbols
MIMO channel
H
C
Fig. 2. The spatial multiplexing system.
In this application example we assume that the transmitted
symbols follow a quaternary phase-shift-keying (QPSK) con-
stellation; i.e., s
C,i
∈ {±1 ± j} for all i. We are interested
in the maximum-likelihood (ML) MIMO detection, which is
optimal in yielding the minimum error probability of detecting
s
C
. It can be shown that the ML problem is equivalent to the
discrete least squares problem
min
s
C
∈{±1±j}
N
ky
C
− H
C
s
C
k
2
, (8)
which is NP-hard [50]. Recent advances in MIMO detection
have provided a practically efficient way of finding a globally
optimal ML solution; viz., the sphere decoding methods [49].
4 APPEARED IN IEEE SP MAGAZINE, SPECIAL ISSUE ON “CONVEX OPT. FOR SP”, MAY 2010
Sphere decoding has been found to be computationally fast
for small to moderate problem sizes; e.g., N ≤ 20. However,
it has been proven that the complexity of sphere decoding is
exponential in N even in an average sense [51].
On the other hand, SDR can be used to produce an approxi-
mate solution to the ML MIMO detection problem in O(N
3.5
)
time, which is polynomial in N. The trick is to turn (8) into
a real-valued homogeneous QCQP. Indeed, by letting
y =
<{y
C
}
={y
C
}
, s =
<{s
C
}
={s
C
}
, H =
<{H
C
} −={H
C
}
={H
C
} <{H
C
}
,
we can rewrite (8) as the following real-valued problem:
min
s∈{±1}
2N
ky − Hsk
2
. (9)
Problem (9) is not a homogeneous QCQP, but we can homog-
enize it as follows:
min
s∈R
2N
, t∈R
kty − Hsk
2
s.t. t
2
= 1, s
2
i
= 1, i = 1, . . . , 2N.
(10)
Problem (10) is equivalent to (9) in the following sense: if
(x
?
, t
?
) is an optimal solution to (10), then x
?
(resp. −x
?
)
is an optimal solution to (9) when t
?
= 1 (resp. t
?
= −1).
With the introduction of the extra variable t, Problem (10) can
then be expressed as a homogeneous QCQP:
min
s∈R
2N
,t∈R
s
T
t
H
T
H −H
T
y
−y
T
H kyk
2
s
t
s.t. t
2
= 1, s
2
i
= 1, i = 1, . . . , 2N.
(11)
Subsequently, SDR can be applied.
We now show some simulation results to illustrate how
well SDR performs in practice. The simulation follows a
standard MIMO setting (see, e.g., [49]), with problem size
(M, N ) = (40, 40). Note that for such a problem size, sphere
decoding is computationally too slow to run in practice. We
tested other benchmarked MIMO detectors, such as the linear
and decision-feedback detectors, and the lattice-reduction-
aided detectors. The results are plotted in Fig. 3. We can see
that SDR provides near-optimal bit error probability, and gives
notably better performance than other MIMO detectors under
test.
In Fig. 3 two performance curves are provided for SDR.
The one labeled ‘SDR with rank-1 approx.’ is the eigenvector
approximation method described in the last section. While this
method is already competitive in performance, the alternative
‘SDR with randomization’ is even more promising. The notion
of randomization will be discussed in Section IV.
Next, we evaluate the computational complexities of the
various MIMO detectors. The results are plotted in Fig. 4. Of
particular interest is the comparison between SDR and optimal
sphere decoding. We see that SDR maintains a polynomial-
time complexity with respect to N. For sphere decoding, the
complexity is attractive for small to moderate N, say N ≤ 16,
but it increases very significantly (exponentially) otherwise.
We conclude this section by pointing out the current ad-
vances of this SDR application. In essence, the promising
performance of SDR MIMO detection in QPSK and binary
PSK (BPSK) has stimulated much interest. That has resulted
0 5 10 15 20 25
10
−4
10
−3
10
−2
10
−1
10
0
Bit Error Probability
ZF
MMSE−DF
LRA−ZF−DF
LRA−MMSE−DF
SDR, with randomization
SDR, with rank−1 approx.
performance lower bound
SNR, in dB
Fig. 3. Bit error probability performance of various MIMO detectors in a
QPSK 40 × 40 MIMO system. ‘ZF’— zero forcing, ‘MMSE’— minimum
mean square error, ‘DF’— decision feedback, ‘LRA’— lattice reduction aided.
‘performance lower bound’ is the bit error probability when no MIMO
interference exists.
5 10 15 20 25 30
10
−4
10
−3
10
−2
10
−1
10
0
10
1
Problem Size N
Average Running Time, in Seconds
ZF
LRA−MMSE−DF
sphere decoding (Schnorr−Euchner)
SDR, with randomization
Fig. 4. Complexity comparison of various MIMO detectors. SNR= 12dB.
in endeavors to extend SDR MIMO detection to other con-
stellations, such as M-ary PSK [20] and M-ary QAM [22],
[23], [52]–[55]. Moreover, treatments for coded MIMO sys-
tems [19], [56] and fast practical implementations [21], [45],
[57] have been considered. On another front, the theoretical
performance of SDR MIMO detection has been analyzed in
various settings. For instance, it has been shown that SDR
can achieve full receive diversity for BPSK [58]. Furthermore,
SDR approximation accuracies relative to the true ML have
been investigated in [59], [60].
IV. RANDOMIZATION AND PROVABLE APPROXIMATION
ACCURACIES
Besides the eigenvector approximation method mentioned
in Section II, randomization is another way to extract an
approximate QCQP solution from an SDR solution X
?
. The
intuitive ideas behind randomization are not difficult to see, yet
the theoretical implications that follow are far from trivial—
many theoretical approximation accuracy results for SDRs are
proven using randomization. To illustrate the main ideas, let
LUO, MA, SO, YE, & ZHANG, SDR OF QUADRATIC OPT. PROBLEMS 5
us consider again the real-valued homogeneous QCQP
min
x∈R
n
x
T
Cx
s.t. x
T
A
i
x D
i
b
i
, i = 1, . . . , m.
(12)
Now, let X ∈ S
n
be an arbitrary symmetric positive semidef-
inite matrix. Consider a random vector ξ ∈ R
n
drawn
according to the Gaussian distribution with zero mean and
covariance X; or ξ ∼ N(0, X) for short. The intuition
of randomization lies in considering the following stochastic
QCQP:
min
X∈S
n
, X0
E
ξ∼N (0,X)
{ξ
T
Cξ}
s.t. E
ξ∼N (0,X)
{ξ
T
A
i
ξ} D
i
b
i
, i = 1, . . . , m,
(13)
where we manipulate the covariance matrix of ξ so that the
expected value of the quadratic objective is minimized and the
quadratic constraints are satisfied in expectation. Interestingly,
through the simple relation X = E
ξ∼N (0,X)
{ξξ
T
}, one can
see that the stochastic QCQP in (13) is equivalent to the SDR
min
X∈S
n
, X0
Tr(CX)
s.t. Tr(A
i
X) D
i
b
i
, i = 1, . . . , m.
(14)
Thus, the stochastic QCQP interpretation of SDR in (13)
provides us with an alternative way to generate approximate
solutions to the QCQP (12). Indeed, after obtaining an optimal
solution X
?
to the SDP (14), we can generate a random
vector ξ ∼ N(0, X
?
) and use it to construct an approximate
solution to the QCQP (12). Note that the specific design
of the randomization procedure is problem-dependent. As an
illustration, let us consider two representative examples.
Example: Randomization in BQP or MIMO detection
For the BQP in (2) or the MIMO detection problem in (11),
a typical randomization procedure is as follows.
Box 2. Gaussian Randomization Procedure for BQP
given an SDR solution X
?
, and a number of randomizations L.
for ` = 1, . . . , L
generate ξ
`
∼ N (0, X
?
), and construct a QCQP-feasible point
˜
x
`
= sgn(ξ
`
). (15)
end
determine `
?
= arg min
`=1,...,L
˜
x
T
`
C
˜
x
`
.
output
ˆ
x =
˜
x
`
?
as the approximate QCQP solution.
In Box 2, the problem dependent part lies in (15), where
we use rounding to generate feasible points from the random
samples ξ
`
. Moreover, we repeat the random sampling L times
and choose the one that yields the best objective.
In the MIMO detection example in Section III, we have seen
that the Gaussian randomization procedure provides quasi-
optimal bit-error-rate performance; see Fig. 3. Here we give
an additional result, plotted in Fig. 5, that shows how the
performance improves with the number of randomizations
L. We see a significant performance gain from L = 1 to
L = 50. The gain becomes smaller for L > 50, approaching
a limit. This shows that randomization provides an effective
approximation for SDR, for sufficient (but not excessive)
number of randomizations.
0 2 4 6 8 10 12 14 16
10
−4
10
−3
10
−2
10
−1
10
0
Bit Error Probability
L= 1
L= 5
L= 20
L= 50
L= 80
L= 120
L= 160
SNR, in dB
Fig. 5. Performance of various numbers of randomizations in MIMO
detection, under the same simulation settings as that in Fig. 3.
Example: Randomization in Problem (3)
This example aims to geometrically illustrate how random-
ization behaves. Consider Problem (3), restated here as
min
x∈R
n
x
T
Cx
s.t. x
T
A
i
x ≥ 1, i = 1, . . . , m,
(16)
where C, A
1
, . . . , A
m
0. Recall that Problem (16) arises
in the context of multicast downlink transmit beamforming.
We set up a numerical example where n = 2, m = 6,
and then generate many random points ξ ∼ N(0, X
?
) to see
how they distribute in space. An instance of this is shown in
Fig. 6. From the distribution of ξ (marked as black ‘·’), one
can see that the covariance matrix X
?
is not of rank one, but
the density is higher over the direction of the globally optimal
QCQP solutions
2
(marked as green ‘∗’). Also, note that the
random samples ξ are not always feasible for (16), but we can
apply a rescaling
x(ξ) =
ξ
p
min
i=1,...,m
ξ
T
A
i
ξ
(17)
to turn them into feasible solutions. We apply the same
rescaling to feasible ξ, too. The rescaled samples x(ξ) are
shown as red ‘o’ in Fig. 6(a). Remarkably, one can see that
there is a significant amount of x(ξ) that lie close to the
optimal QCQP solutions.
A practical randomization procedure for Problem (16)
is essentially identical to that presented in Box 2, except
that (15) is replaced by (17). Such a procedure has been
empirically found to provide promising approximations for
the multicast downlink transmit beamforming application and
its variations, like the MIMO detection application. Readers
are referred to [27], [61] for the results.
2
In this example, the globally optimal QCQP solutions were obtained by
a fine grid search on R
2
. Such an exhaustive search would be prohibitive
computationally for general R
n
.