pubs.acs.org/jmc
r
XXXX American Chemical Society
J. Med. Chem. XXXX, XXX, 000–000 A
DOI: 10.1021/jm901137j
New Substructure Filters for Removal of Pan Assay Interference Compounds (PAINS) from Screening
Libraries and for Their Exclusion in Bioassays
Jonathan B. Baell*
,†,‡
and Georgina A. Holloway
†,‡
†
The Walter and Eliza Hall Institute of Medical Research, 1G Royal Parade, Parkville, Victoria 3052, Australia and
‡
Cancer Therapeutics-CRC
P/L, 4 Research Avenue, La Trobe R&D Park, Bundoora, Victoria 3086, Australia
Received July 31, 2009
This report describes a number of substructural features which can help to identify compounds that
appear as frequent hitters (promiscuous compounds) in many biochemi cal high throughput screens. The
compounds identified by such substructural features are not recognized by filters commonly used to
identify reactive compounds. Even though these substructural features were identified using only one
assay detection technology, such compounds have been reported to be active from many different
assays. In fact, these compounds are increasingly prevalent in the literature as potential starting points
for further exploration, whereas they may not be.
Introduction
High throughput screening (HTS
a
) is a key discipline
undertaken by pharmaceutical companies as part of success-
ful drug discovery. It is inevitable that hits from HTS cam-
paigns comprise predominantly false positives among the real
hits-if there are any.
1
Compounds can be regarded as false
positives for a number of reasons, for example, those that
interfere in binding interactions by forming aggregates,
2-5
those that are protein-reactive,
6,7
or those that directly inter-
fere in assay signaling. Aggregate formation and related
behavior can be minimized by the inclusion of surfactant in
the primary screening protocol,
2,8
while the most reactive
compounds can be largely weeded out before purchase by
the use of well publicized functional group filters.
9-11
How-
ever, it is clear that much about the nature of protein-reactive
compounds remains to be known. With respect to the third
cause of false positives, interference in assay signaling, it is not
always clear which compounds might interfere with the
primary screening assay technology until they are purchased
and tested in an appropriately designed counter screen for
assay interference.
The current chemical biology renaissance has seen HTS
gain increasing traction in academic laboratories as a viable
means to furnish tool compounds as well as starting points for
therapeutic development. In 2003, our Institute established a
screening library of 93212 compounds from commercial
vendors. While now there are many dozens of such initiatives,
at that time we were one of only a handful of academic
laboratories worldwide to do so on this scale. The guiding
philosophy behind our selection criteria was that being
espoused by proponents of lead-like simplicity rather than
drug-like complexity for furnishing more optimizable screen-
ing hits as better starting points for drug development that
better cover diversity space.
12,13
We have commented on this
elsewhere.
14,15
We also removed all compounds more than
85% similar to any other compound as determined by the
Tanimoto coefficient, as we found this to be the most simple
and yet still reasonable way of removing large numbers of
similar analogues for highly populated chemistries while still
retaining a skeleton structure-activity relationship (SAR) set.
We selected from only four large vendors, as we determined
that there was significant overlap between many of the 15
vendor databases under investigation, a finding that has since
been confirmed.
16
Vendor selection was also influenced by
independently obtained referees reports. We largely adhered
to published guidelines
9-11
for removal of inappropriately
reactive functional groups, as summarized in Table 1 (see
Table S1 in Supporting Information (SI) for a more complete
listing). However, some groups such as aminothiophenes,
azo, anilines, oximes, hydrazones, thioethers, thiocarbonyls,
ketones, esters, catechols, quinones, and some R,β-unsatu-
rated carbonyl groups we retained. Where inclusion of these
groups in screening is advised against,
9-11,16
it is generally on
the basis of poor development potential or toxicity or, in the
case of esters, hydrolytic liability on storage. Our reasoning
for retaining these groups was that if such a compound
furnished the only hit for a given target, unwanted features
could be removed during medicinal chemistry optimization.
Furthermore, we anticipated that thioether, hydrazone, and
ester linkages, for example, might link potential binding
elements that would otherwise not be represented. We would
be concerned if such compounds also interfered in HTS
assays, but at the stage of library implementation we had no
reason to suspect this. We note importantly that analytical
techniques have since confirmed compound integrity for more
than 95% of furnished screening hits.
To date, we have run more than 40 primary screening
campaigns against enzymes, ion channels, protein-protein
*To whom correspondence should be addressed. Phone: þ61 3 9345
2108. Fax: þ61 3 9345 2211. E-mail: jbaell@wehi.edu.au.
a
Abbreviations: AMH, Australian Medicines Handbook; CTX, CRC
for Cancer Therapeutics; DABCO, 1,4-diazabicyclo[2.2.2]octane; FP,
fluorescence polarization; HTRF, homogeneous time-resolved fluores-
cence; HTS, high throughput screening; MDDR, MDL Drug Data
Report; MOA, mechanism of action; PAINS, pan assay interference
compounds; SAR, structure-activity relationship; WEHI, Walter and
Eliza Hall Institute of Medical Research.
B Journal of Medicinal Chemistry, XXXX, Vol. XXX, No. XX Baell and Holloway
interactions, and whole cells. However, there are significant
numbers of false positives that continue to incur a resource
penalty. We are currently expanding our screening library to
250000. In so doing, we wished to incorporate selection
criteria built on the knowledge garnered from previous HTS
campaigns. In particular, we wanted to develop substructure
filters that efficiently encoded for problematic compounds in
order to exclude these and their analogues from the pending
bulk purchase.
We reveal herein the results of our analysis and the proble-
matic structural classes that we have identified. We show that
these compounds are prevalent in the screening literature and
often painted as valid development starting points. However,
we conclude that they are probably protein-reactive and poor
drug development choices. Finally, we fully disclose our
structural filters in readily transferable Sybyl line notation
(sln) for others to utilize.
Results
The data from six HTS campaigns were selected for
analysis (SI Table S2). All HTS campaigns discussed in this
paper displayed an acceptable signal-to-noise ratio as
judged by the Z
0
values
17
being great er than 0.5 in each
assay event. The six HTS campaigns we selected employed
AlphaScreen technology (PerkinElmer), which we have
found to be particularly robust and reproducible. In these
campaigns, rigorous compound triaging through counter-
screens, secondary screens, and finally direct binding assess-
ment by surface plasmon resonance is routinely undertaken.
However, for the purposes of this current study, we delibe-
rately selected the primary hit set (single point inhibition) as
that which represented the least-processed set for each assay.
Rather, we controlled for experimental aberration through
the use of multiple (six) sets of primary screening hits. Primary
hit selection criteria are given in SI Table S2. We interrogated
the primary screening hits as follows. All library compounds
were binned according to the number of assays in which they
registered as hits according to those criteria for each particular
assay. As a general rule, compound screening concentrations
of between 10 and 30 μM were deployed. The results are
shown in Table 2.
It can be seen that, reassuringly, the majority of the library
compounds are clean, with 73164 registering a score of 0 and
a further 12 077 only registering as a hit in one assay event.
However, there are 7972 compounds that hit two or more
assays, of which 2062 hit four or more. The latter in
particular are clearly potentially highly problematic and
form the basis on which we derived our assay interference
substructure filters. We wished to define substructures that
selectively recognized the most problematic compounds in
well-defined classes. These would form the basis of filters
when selecting vendor compounds for our library expansion.
To rationally achieve this, we needed to define first when a
compound was considered problema tic and second how we
would quantify selective removal of a problematic class, that
is, to define what an acceptable number would be of clean
compounds that might be concomitantly removed by a
substructure filter intended to remove problematic com-
pounds.
For the first requirement, we analyzed all validated hits
(i.e., true and confirmed specific binders) from around 30
target-based HTS campaigns in order to find that which
happened to also register as a hit in the greatest number
(if any) of the six HTS campaigns under scrutiny here. We
found that one of these gave inhibition readouts of 74%, 58%,
81%, and 67% in HTS campaigns A, B, D, and E, respec-
tively. For campaigns C and F, inhibition was measured at
<50%, i.e., below that which would define a compound as a
hit. A screening concentration of test compound of 50 μMwas
used for all HTS campaigns except for HTS campaign D,
where the relevant concentration was 25 μM (see SI Table S3).
Limited investigation using surface plasmon resonance sug-
gested that this compound had a tendency to bind nonspeci-
fically to proteins (data not shown). Conversely, this
compound was found to bind reversibly and specifically to
another target of interest to us (not discussed here) with an
IC
50
of around 10 μM. This compound was subsequently
optimized to single digit nanomolar levels of activity to this
same target. The point of this discussion is to illustrate that an
optimizable screening hit may also weakly interfere in other
assays under our screening conditions. We do not attribute a
high assay interference rate to the assay technology because,
under our optimized conditions (see SI Table S2), our experi-
ence is that the AlphaScreen platform performs favorably
compared with other platforms such as fluorescence polariza-
tion (FP). Rather, a high assay interference rate is better
attributed to the relatively high screening concentrations
(25-50 μM) that we are prepared to employ, governed by
the philosophy that weaker hits are acceptable if small and
highly optimizable. We did not want to be so harsh in our
definition of a problematic compound that we also removed
potentially useful or proven screening hits.
These considerations finally led to the definition that a
problematic compound was that with a count of 4 or more and
whose target inhibition was greater than 85% in at least one
HTS campaign or greater than 80% in at least two HTS
campaigns, at the screening concentration tested. It is not
clear to us that there is any statistically validated or less
arbitrary alternative to define a cutoff point for when a
compound is considered to be problematic.
Table 1. Functional Groups Commonly Recommended as Unsuitable for Screening Libraries, Partitioned into Those That We Also Removed and
Some That We Chose Not to Remove during the Establishment of the Inaugural 2003 93K WEHI HTS Library
removed alkyl halides, acid halides, alkyl sulfonates, anhydrides, peroxides, isocyanates, triflates, positively charged carbon/halogen/phosphorus/
sulfur, any heterohalide, carbodiimide, acyl cyanides, sulfonyl cyanides, disulfides, thiols, epoxides, aziridines, betalactones,
betalactams, labile esters, aldehydes, certain imines, phosphate/sulfate/phosphonate/sulfonate esters, reactive michael acceptors,
ketenes, oxoniums, carbamic acids, boronic acids, primary hydrazines/oxyamines, P-N, P-S, cyclohexadienes, activated sulfonyl
(hetero)aryl halides, fluoropyridines, nitro
not removed aminothiophenes, ketones, esters, hydrazones, oximes, thiocarbonyl, thioethers, R,β-unsaturated amides, azo, anilines, quinones,
catechols
Table 2. Breakdown of Primary Screening Hits According to the
Number of HTS Screening Campaigns in Which They Registered As a
Hit
count
a
6 5 4 3 2 1 0 total
no. compds 362 785 915 1220 4689 12077 73164 93212
a
Number of assays hit (out of six possible).
Article Journal of Medicinal Chemistry, XXXX, Vol. XXX, No. XX C
It is of course possible that some potentially useful com-
pounds could still fall under our definition of a problematic
compound and therefore be inappropriately recognized by our
filters: for example, dyes that simply interfere with the assay
readout technology but could in principle bind the right target
protein potently and selectively, should that protein target be
screened against, or compounds like our proven hit in SI Table
S3 that can bind weakly but in a specific manner to several
different proteins but that could be rendered selective for any
given protein with appropriate optimization (somewhat akin
to fragment screening). However, we were comfortable that
our cutoff criteria would identify predominantly nuisance
compounds and that the benefits of this would outweigh the
drawbacks associated with the potential loss of a useful hit.
To start defining substructure filters, we initially scrutinized
the 362 most problematic structures that hit all six assays and
were able to group them into around 30 broad classes (see SI
Table S4). We selected half a dozen of the most readily defined
and recognizable substructures, such as quinones, rhodanines,
and 2-aminothiophenes, filtered these out from our library and
analyzed the counts. We did likewise for six benign moieties
such as the amide group that we saw as unlikely to be linked to
assay interference chemistry. We observed a stark contrast
between the two classes when we totalled the number of com-
pounds in the filtered set that hit two to six assays and expressed
this as a percentage of those that were clean in the same filtered
set. Whereas the values of the former ranged from 41 to 625%,
the latter ranged from 8 to 18%, similar to the WEHI library
(11%, calculated from Table 2). These data are summarized in
Table 3 and the data more fully disclosed in SI Table S5.
On this basis, we defined that any given substructure filter
encoded acceptably for problematic compounds if, from those
compounds recognized by the filter and removed, the propor-
tion of compounds that hit two to six assays was greater than
30% of those that were clean. We term this our enrichment
value.
In this way, we continued to evaluate all problematic
structures, augmenting substructural descriptions or defining
new ones, until all problematic compounds were accounted
for in recognition filters. Toward the end we observed that
considerable numbers of singletons began to appear. In the
first instance, we did not evaluate the enrichment value of
individual substructure filters but simply targeted 30% as the
required overall enrichment value.
The effect of passing our resulting alpha filters through our
library is shown in Table 4 and as initially aimed for the
enrichment value (32%) exceeded our target of 30%.
In the next phase of filter refinement, we analyzed indivi-
dual substructure filters to ensure that each of these selectively
removed problematic compounds according to our enrich-
ment target value of 30%. We did this as follows. We took all
substructure filters from our alpha analysis that encoded for
150 compounds or more in the one family (filter family A) and
assessed each of these for their enrichment value. We subse-
quently removed any of those substructure filters from further
consideration where the enrichment value was below 25%.
For all others, we compared clean structures with problematic
structures and rationally revised each substructure filter so
that enrichment was maximized to the best of our ability and
was at least 30%. We then partitioned our library by removing
from further consideration all compounds that were encoded
in this filter family A. We then repeated the same procedure
for all substructure filters that encoded between 15 and 149
compounds (filter family B) and subsequently all those that
encoded 1-14 compounds (filter family C). In this way, some
groups that we had initially identified as potentially proble-
matic turned out not to be. For example, 2-alkenylfurans are
conspicuous in the most problematic compounds in SI Table
S4 but are not themselves problematic. Likewise, sulfur is
prevalent in frequent hitters but only is problematic in the
context of specific functional groups.
The effect of passing this, our refined filter, through our
library is shown in Figure 1. Here, it can be seen that filter
family A removes 4703 compounds, filter family B 2196
compounds, and filter family C 1186 compounds.
In Table 5, is it seen that while in total 480 substructures were
required to filter out all assay interference compounds, the
majority of these compounds (4703, filter family A) are encoded
by only 16 substructures, and a further 2196 are encoded by 55
substructures (family filter B). However, 409 substructures in
family filter C only account for 1186 compounds in total.
These refined filters are fully provided in the SI. Each filter
family is provided as a separate text file of substructure defini-
tions(SITablesS6-S8). Substructures are written in Sybyl line
notation (sln) and so are Sybyl-ready for the dbslnfilter algo-
rithm. We also provi de these graphically as 2D structures,
grouped according to structural class (SI Figure S1) as broadly
defined by the name given to each substructure. We also
provide the count and enrichment calculation in SI Table S9
for each of the 174 individual substructure filters whose family
size contains four or more compounds.
The effect of passing our refined filters through our com-
pound library are shown in Table 6.
Table 3. Comparison Showing the Greater Proportion of HTS Cam-
paigns Hit by Problematic Compounds than Clean Compounds
category substructure
proportion hitting 2-6
screens compared with
those hitting no screens (%)
clean amide 8
clean 2-aminopyridine 10
clean benzothiazole 14
clean chlorophenyl 11
clean aromatic N 16
problematic rhodanine-like 41
problematic 2-aminothiophene 43
Table 4. Effect of Filtering the Inaugural WEHI-Bio21 HTS Library through the Alpha Version of Our Filters
count
a
6 5 4 3 2 1 0 total enrichment
b
(%)
no. of compds in problematic
compds file
362 785 809 835 2094 3846 15157 23888 32
no. of compds
retained
0 0 106 385 2595 8233 58005 69324 5.3
total 362 785 915 1220 4689 12079 73162 93212
a
Number of assays hit (out of six possible).
b
The number of compounds that hit between two and six assays expressed as a percentage of those that hit
no assays.
D Journal of Medicinal Chemistry, XXXX, Vol. XXX, No. XX Baell and Holloway
Here, it is clear by looking at the relative enrichment levels
in the clean and problematic compounds that substantial
improvement has been made over our alpha filters
(Table 4), with substantially fewer clean compounds present
in the problematic compounds file, leading to an enrichment
value of 110%. There is much hidden information in these
data. For example, while compounds with counts of 1-3did
not feature whatsoever in our filter definition process, they are
selectively recognized by the problematic compound filters: in
the problematic file the percentage of compounds with a count
of 3 relative to those with a count of 1 or 0 is 40% and 17%,
respectively. The corresponding figures for the clean file are
6.2% and 1.7%, respectively. While a compound with a count
of 3 is relatively clean by our criteria, such compounds are
relatively enriched in the problematic compound file. By
definition, compounds in the problematic compound file must
be closely related to one or more compounds that are hits in 4
or more HTS campaigns. In other words, a relatively clean hit
with a count of 3 is more plausibly a genuine hit if not
recognized by our problematic compound filters. This is a
useful tool for prioritizing hits from our HTS library.
We were interested in the relationship of compound source
to problematic compounds. Shown in Table 7 are the results of
processing a number of different compound collections. First,
we show that the source of our library compounds does
influence the percentage of problematic compounds; the
vendors that represent historical and combinatorial collections
Figure 1. Schematic effect of passing our filter families through the
WEHI 93K HTS library.
Table 5. Number of Substructures Required to Remove All Proble-
matic Compounds and Number of Compounds Per Substructure
grouping
no. of compds per
substructure filter
no. of substructures
in filter family
no. of compds
(duplicates)
a
filter family A >149 16 4703 (230)
filter family B 15-149 55 2196 (52)
filter family C 1-14 409 1186 (6)
a
Duplicates represent those compounds that contain more than one
problematic substructure.
Table 6. Breakdown of Primary Screening Hits According to the Number
of HTS Screening Campaigns in Which They Registered As a Hit, Divided
into the Filtered File (Problematic Compounds) and Those Retained
count
a
6 5 4 3 2 1 0 total
enrichment
b
(%)
no. of compds
in problematic
compds file
362 785 724 552 1104 1372 3186 8085 110
no. of compds
retained
0 0 191 668 3585 10707 69976 85127 5.1
total 362 785 915 1220 4689 12079 73162 93212
a
Number of assays hit (out of six possible).
b
The number of
compounds that hit between two and six assays expressed as a percen-
tage of those that hit no assays.
Table 7. Percentages of Compounds in a Variety of Compound Collec-
tions That Are Recognized by Our Filters for Problematic Compounds
entry compd collection
no. of
compds
problematic
compds
a
(%)
WEHI 93K library
b
1 vendor A (historical/combinatorial) 36370 13
2 vendor B (historical/combinatorial) 35606 9.6
3 vendor C (custom-made) 13408 4.0
4 vendor D (custom-made) 14893 6.0
MDDR 2008.1
5 biological testing 169066 5.1
6 preclinical 11645 5.8
7 phase I 931 4.8
8 phase II 1372 4.2
9 phase III 415 3.4
10 launched 1808 5.0
other marketed drugs
11 AMH 2008
c
743 6.5
12 CNS set
d
142 2.8
independent library
13 AlphaScreen primary hit set
e
5237 51
WEHI HTS campaigns primary hit set
14 campaign A 3006 68
15 campaign B 4086 53
16 campaign C 3145 72
17 campaign D 746 81
18 campaign E 9,309 33
19 campaign F 14,745 28
current vendors (June 2009)
f
20 vendor A 1165361 4.6
21 vendor B 239674 8.7
22 vendor D 61623 4.6
23 vendor E 207573 6.5
24 vendor F 619514 5.2
25 vendor G 392499 11.6
26 vendor H 289,552 5.0
27 vendor I 449998 10.8
28 vendor J 193379 8.2
a
Those in total filtered out by filter families A-C.
b
There are 100277
compounds in total arising from 7065 duplicate compounds.
c
Australian
Medicines Handbook (2008).
d
A set of proprietary CNS active com-
pounds. The compounds filtered out were thiothixene, amitriptyline,
nortriptyline, and frovatriptan.
e
Primary hits from screening a colla-
borator’s library.
f
Vendors A, B, and D (respective entries 20, 21, and 22)
correspond to the same respective vendors A, B, and D for the WEHI
93K HTS library (respective entries 1, 2, and 4).
Article Journal of Medicinal Chemistry, XXXX, Vol. XXX, No. XX E
tend to contain a greater proportion of problematic com-
pounds. Hence, the percentages here are 13% and 9.6% for
vendors A and B, respectively, compared with 4% and 6% for
vendors C and D. The latter vendors describe their approach
to library design as being more tailor-made than combinator-
ial, suggesting a link with problematic compounds and facile
chemistry.
We also analyzed MDDR 2008.1, and here it is clear that a
continuous percentage of compounds register as problematic
compounds from preclinical to launched categories without
apparently trending upward or downward. Similarly, 48
compounds (6.5%) out of 743 from the Australian Medicines
Handbook (2008) are recognized by our problematic com-
pound filters. These percentages are, however, relatively small
and similar to those noted in tailor-made vendor libraries.
Prevalent in these compounds are quinones/acridine-based
cytotoxics, but there are also neuroleptics, dopamine-like
catechols, and occasional steroids, dyes, and phenylmorpho-
lines (see SI Figure S2).
A set of CNS compounds revealed only four (2.8%) as
being recognized by our problematic compound filters
(entry 12).
To ascertain that assay interference compounds recognized
by our filters were not peculiar to our library, we tested the
effect of passing our filters through another primary hit set
that we had obtained from a primary screen of a commercial
collaborator’s HTS library using AlphaScreen technology. As
shown in Table 7 (entry 13), in marked contrast to drug-like
compounds, this screening library primary hit set registered a
substantial number of compounds (51%) as being recognized
by our problematic compound filters.
We also passed our filters back through our original
primary hit sets from the six HTS campaigns under investiga-
tion here (SI Table S2), and as shown in Table 7, as would be
hoped, a large percentage of primary hits are removed by our
filters, the highest being 81% for HTS campaign D (entry 17).
The two lowest percentages of 33% and 28%, respectively, are
represented by HTS campaigns E and F, which used a
different anchoring technology, causing a high hit rate with
dilution of the primary screening set due to interference by
mildly chelating compounds (see SI Table S2).
Finally, we passed our filters through the databases of nine
popular current vendors. Also shown in Table 7, the percen-
tage of problematic compounds varies from a low of 4.6% for
vendor A (entry 20) to a significant high of 11.6% for vendor
G (entry 25). Intriguingly, vendor A in 2003 contained a
significantly higher percentage of these problematic com-
pounds (13%, entry 1), suggestive of a deliberate effort to
minimize the numbers of such compounds.
Significantly, our filters would have removed problematic
compounds shown in Figure 2 that have incurred significant
costs through some medicinal chemistry follow up that ulti-
mately failed due to the lack of robust SAR. We term these cul
de sac compounds. We note that these compounds often
showed signs of early SAR and selectivity when tested in
appropriately designed counter screens so that we embarked
Figure 2. Problematic cul de sac compounds that have incurred wasted resources through being followed up to varying degrees at our Institute.
We have found chromones such as 5 to be highly susceptible to nucleophilic attack at the 2-position, while β-amino sulfones (and ketones) such
as 2 readily form reactive retro Michael alkenes. Compounds 6-9 are also susceptible to attack by biologically relevant nucelophiles. The other
compounds are problematic for reasons that are either discussed in the text or remain unknown.
