Statistical Applications in Genetics
and Molecular Biology
Volume 3, Issue 1 2004 Article 3
Linear Models and Empirical Bayes Methods
for Assessing Differential Expression in
Microarray Experiments
Gordon K. Smyth
∗
∗
Walter and Eliza Hall Institute, smyth@wehi.edu.au
Copyright
c
2004 by the authors. All rights reserved. No part of this publication may be re-
produced, stored in a retrieval system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise, without the prior written permission of the
publisher, bepress, which has been given certain exclusive rights by the author. Statistical Applica-
tions in Genetics and Molecular Biology is produced by The Berkeley Electronic Press (bepress).
http://www.bepress.com/sagmb
Linear Models and Empirical Bayes Methods
for Assessing Differential Expression in
Microarray Experiments
∗
Gordon K. Smyth
Abstract
The problem of identifying differentially expressed genes in designed microarray experiments
is considered. Lonnstedt and Speed (2002) derived an expression for the posterior odds of differen-
tial expression in a replicated two-color experiment using a simple hierarchical parametric model.
The purpose of this paper is to develop the hierarchical model of Lonnstedt and Speed (2002) into
a practical approach for general microarray experiments with arbitrary numbers of treatments and
RNA samples. The model is reset in the context of general linear models with arbitrary coefficients
and contrasts of interest. The approach applies equally well to both single channel and two color
microarray experiments. Consistent, closed form estimators are derived for the hyperparameters
in the model. The estimators proposed have robust behavior even for small numbers of arrays and
allow for incomplete data arising from spot filtering or spot quality weights. The posterior odds
statistic is reformulated in terms of a moderated t-statistic in which posterior residual standard
deviations are used in place of ordinary standard deviations. The empirical Bayes approach is
equivalent to shrinkage of the estimated sample variances towards a pooled estimate, resulting in
far more stable inference when the number of arrays is small. The use of moderated t-statistics has
the advantage over the posterior odds that the number of hyperparameters which need to estimated
is reduced; in particular, knowledge of the non-null prior for the fold changes are not required.
The moderated t-statistic is shown to follow a t-distribution with augmented degrees of freedom.
The moderated t inferential approach extends to accommodate tests of composite null hypotheses
through the use of moderated F-statistics. The performance of the methods is demonstrated in a
simulation study. Results are presented for two publicly available data sets.
KEYWORDS: microarrays, empirical Bayes, linear models, hyperparameters, differential ex-
pression
∗
Walter and Eliza Hall Institute, 1G Royal Parade, Melbourne 3050, Australia,
smyth@wehi.edu.au
1 Introduction
Microarrays are a technology for comparing the expression profiles of genes on a genomic
scale across two or more RNA samples. Recent reviews of microarray data analysis
include the Nature Genetics supplement (2003), Smyth et al (2003), Parmigiani et
al (2003) and Speed (2003). This paper considers the problem of identifying genes
which are differentially expressed across specified conditions in designed microarray
experiments. This is a massive multiple testing problem in which one or more tests
are conducted for each of tens of thousands of genes. The problem is complicated by
the fact that the measured expression levels are often non-normally distributed and
have non-identical and dependent distributions between genes. This paper addresses
particularly the fact that the variability of the expression values differs between genes.
It is well e stablished that allowance needs to be made in the analysis of microarray
experiments for the amount of multiple testing, perhaps by controlling the familywise
error rate or the false discovery rate, even though this reduces the power available
to detect changes in expression for individual genes (Ge et al, 2002). On the other
hand, the parallel nature of the inference in microarrays allows some compensating
possibilities for borrowing information from the ensemble of genes which can assist in
inference about each gene individually. One way that this can be done is through the
application of Bayes or empirical Bayes methods (Efron, 2001, 2003). Efron et al (2001)
used a non-parametric empirical Bayes approach for the analysis of factorial data with
high density oligonucleotide microarray data. This approach has much p otential but can
be difficult to apply in practical situations especially by less experienced practitioners.
L¨onnstedt and Sp e ed (2002), considering replicated two-color microarray experiments,
took instead a parametric empirical Bayes approach using a simple mixture of normal
models and a conjugate prior and derived a pleasingly simple expression for the posterior
odds of differential expression for each gene. The posterior odds expression has proved
to be a useful means of ranking genes in terms of evidence for differential expression.
The purpose of this paper is to develop the hierarchical model of L¨onnstedt and
Speed (2002) into a practical approach for general microarray experiments with arbi-
trary numbers of treatments and RNA samples. The first step is to reset it in the
context of general linear models with arbitrary coefficients and contrasts of interest.
The approach applies to both single channel and two color microarrays. All of the
commonly used microarray platforms such as cDNA, long-oligos and Affymetrix are
therefore accommodated. The second step is to derive consistent, closed form estima-
tors for the hyperparameters using the marginal distributions of the observed statistics.
The estimators proposed here have robust behavior even for small numbers of arrays and
allow for incomplete data arising from spot filtering or spot quality weights. The third
step is to reformulate the posterior odds statistic in terms of a moderated t-statistic
in which p osterior residual standard deviations are used in place of ordinary standard
deviations. This approach makes explicit what was implicit in L¨onnstedt and Speed
(2002), that the hierarchical model results in a shrinkage of the gene-wise residual sam-
ple variances towards a common value, resulting in far more stable inference when the
number of arrays is small. The use of moderated t-statistic has the advantage over
1
Smyth: Empirical Bayes Methods for Differential Expression
Published by The Berkeley Electronic Press, 2004
the posterior odds of reducing the number of hyperparameters which need to estimated
under the hierarchical model; in particular, knowledge of the non-null prior for the fold
changes are not required. T he moderated t-statistic is shown to follow a t-distribution
with augmented degrees of freedom. The moderated t inferential approach extends
to accommodate tests involving two or more contrasts through the use of moderated
F -statistics.
The idea of using a t-statistic with a Bayesian adjusted denominator was also pro-
posed by Baldi and Long (2001) who developed the useful cyberT program. Their work
was limited though to two-sample control versus treatment designs and their model did
not distinguish between differentially and non-differentially expressed genes. They also
did not develop consistent estimators for the hyperparameters. The degrees of freedom
associated with the prior distribution of the variances was set to a default value while
the prior variance was simply equated to locally pooled sample variances.
Tusher et al (2001), Efron et al (2001) and Broberg (2003) have used t statistics
with offset standard deviations. This is similar in principle to the moderated t-statistics
used here but the offset t-statistics are not motivated by a model and do not have an
associated distributional theory. Tusher et al (2001) estimated the offset by minimizing
a coefficient of variation while Efron et al (2001) used a percentile of the distribution
of sample standard deviations. Broberg (2003) considered the two sample problem and
proposed a computationally intensive method of de termining the offset by minimizing a
combination of estimated false positive and false negative rates over a grid of significance
levels and offsets. Cui and Churchill (2003) give a review of test statistics for differential
expression for microarray experiments.
Newton et al (2001), Newton and Kendziorski (2003) and Kendziorski et al (2003)
have considered empirical Bayes models for expression based on gamma and log-normal
distributions. Other authors have used Bayesian methods for other purposes in mi-
croarray data analysis. Ibrahim et al (2002) for example propose Bayesian models with
correlated priors to model gene expression and to classify between normal and tumor
tissues.
Other approaches to linear models for microarray data analysis have been described
by Kerr et al (2000), Jin et al (2001), Wolfinger et al (2001), Chu et al (2002), Yang
and Speed (2003) and L¨onnstedt et al (2003). Kerr et al (2000) propose a single linear
model for an entire microarray experiment whereas in this paper a separate linear model
is fitted for each gene. The single linear model approach assumes all equal variances
across genes whereas the current paper is designed to accommodate different variances.
Jin et al (2001) and Wolfinger et al (2001) fit separate models for each gene but model
the individual channels of two color microarray data requiring the use of mixed linear
models to accommo date the correlation between observations on the same spot. Chu
et al (2002) propose mixed models for single channel oligonucleotide array experiments
with multiple probes per gene. The methods of the current paper assume linear models
with a single component of variance and so do not apply direc tly to the mixed model
approach, although ideas similar to those used here could be developed. Yang and
Speed (2003) and L¨onnstedt et al (2003) take an linear modeling approach similar to
that of the current paper.
2
Statistical Applications in Genetics and Molecular Biology, Vol. 3 [2004], Iss. 1, Art. 3
http://www.bepress.com/sagmb/vol3/iss1/art3
A B A B
A
Ref
B
A B
C
(a) (b)
(c) (d)
Figure 1: Example designs for two color microarrays.
The plan of this paper is as follows. Section 2 explains the linear modelling approach
to the analysis of designed experiments and specifies the response model and distribu-
tional assumptions. Section 3 sets out the prior assumptions and defines the posterior
variances and moderated t-statistics. Section 4 derives marginal distributions under the
hierarchical mode l for the observed statistics. Section 5 derives the posterior odds of
differential expression and relates it to the t-statistic. The inferential approach based
on moderated t and F statistics is elaborated in Section 6. Section 7 derives estimators
for the hyperparameters. Section 8 compares the estimators with earlier statistics in a
simulation study. Section 9 illustrates the methodology on two publicly available data
sets. Finally, Section 10 makes some remarks on available software.
2 Linear Models for Microarray Data
This section describes how gene-wise linear models arise from experimental designs and
states the distributional assumptions about the data which will used in the remainder
of the paper. The design of any microarray experiment can be represented in terms of
a linear model for each gene. Figure 1 displays some examples of simple designs with
two-color arrays using arrow notation as in Kerr and Churchill (2001). Each arrow
represents a microarray. The arrow points towards the RNA sample which is labelled
red and the sample at the base of the arrow is labelled green. The symbols A, B and C
represent RNA sources to be compared. In experiment (a) there is only one microarray
which compares RNA sample A and B. For this experiment one can only compute the
log-ratios of expression y
g
= log
2
(R
g
)−log
2
(G
g
) where R
g
and G
g
are the red and green
intensities for gene g. Design (b) is a dye-swap experiment leading to a very simple
linear model with responses y
g1
and y
g2
which are log-ratios from the two microarrays
and design matrix
X =
1
−1
!
.
3
Smyth: Empirical Bayes Methods for Differential Expression
Published by The Berkeley Electronic Press, 2004
