Jointly published by Akadémiai Kiadó, Budapest Scientometrics, Vol. 73, No. 1 (2007) 53–78
and Springer, Dordrecht DOI: 10.1007/s11192-007-1681-4
Received January 22, 2007
Address for correspondence:
F
ÉLIX MOYA-ANEGÓN
University of Granada, Library and Information Science Faculty
Campus Cartuja, 18071 Granada, Spain
E-mail: felix@ugr.es
0138–9130/US $ 20.00
Copyright © 2007 Akadémiai Kiadó, Budapest
All rights reserved
Coverage analysis of Scopus:
A metric approach
FÉLIX DE MOYA-ANEGÓN, ZAIDA CHINCHILLA-RODRÍGUEZ,
B
ENJAMÍN VARGAS-QUESADA, ELENA CORERA-ÁLVAREZ,
F
RANCISCO JOSÉ MUÑOZ-FERNÁNDEZ, ANTONIO GONZÁLEZ-MOLINA,
V
ICTOR HERRERO-SOLANA
SCIMAGO Research Group, University of Granada, Library and Information Science Faculty,
Granada (Spain)
Our aim is to compare the coverage of the Scopus database with that of Ulrich, to determine
just how homogenous it is in the academic world. The variables taken into account were subject
distribution, geographical distribution, distribution by publishers and the language of publication.
The analysis of the coverage of a product of this nature should be done in relation to an accepted
model, the optimal choice being Ulrich’s Directory, considered the international point of reference
for the most comprehensive information on journals published throughout the world. The results
described here allow us to draw a profile of Scopus in terms of its coverage by areas – geographic
and thematic – and the significance of peer-review in its publications. Both these aspects are
highly pragmatic considerations for information retrieval, the evaluation of research, and the
design of policies for the use of scientific databases in scientific promotion.
Introduction
To date the Thomson Scientific citation databases have stood out from the rest on
two grounds: their multidisciplinarity and their international coverage. Comprising
approximately 8,700 scientific journals in their Web of Science version, they have
become the main tool for information retrieval and for science evaluation studies
worldwide. Secondly, particular characteristics concerning information related to
F. DE MOYA-ANEGÓN et al.: Coverage analysis of Scopus
54 Scientometrics 73 (2007)
research quality and its impact in the scientific community have led the governments of
some countries to show specific interest in this tool as a means of evaluation, requiring
researchers to have published in journals included in these databases – especially in
those of a high impact factor – and covered by the Journal Citation Report (JCR).
Yet recently (November 2004) an alternative tool has appeared on the information
market: Scopus, created by Elsevier. It is now the largest database for multidisciplinary
scientific literature existing on the market. This navigation tool is updated daily and
includes summaries and references cited since 1966, from more than 14,000 journals
(16,000 if non-active journals are included) and from all areas of knowledge. Scopus
offers a complete package of services aside from its bibliographic contents. Its interface,
for instance, was designed to provide a global view of results, enabling researchers to
quickly identify both what is relevant to them in particular along with the current
overall trends in their fields of study. In searching for citations and references, Scopus
allows connections by disciplines; it achieves high levels of precision when matching
references to summaries; it offers links to the full text, to Open Access sources and
library catalogues, to exclusively electronic publications and to cited articles from the
results, summary and references lists; and researchers can apply for new contents,
sources or document types, even modifying the configurations and personalising their
choices. The literature already includes research studies that analyse the characteristics
of this product (J
ACSO, 2004; CODINA, 2005; LAGUARDIA, 2005).
Meanwhile, comparative studies are appearing on the horizon as well, looking at the
coverage, interface accessibility, usability, price, etc., of Scopus versus ISI (D
EIS &
G
OODMAN, 2004; JACSO, 2004; LAGUARDIA, 2005). It would appear that Scopus and
ISI-WOS have entered head-on competition (P
ICKERING, 2004) and any comparison of
the two products calls for utmost care as well as methodological consistency. Previous
authors have compared Ulrich with ISI databases: B
RAUN et al. (2000) with SCI, and
A
RCHAMBAULT et al. (2005) with SSCI and A&H. Ulrich’s Directory is clearly a
worldwide point of reference for the most comprehensive information on journals
published the world over.
However, the aim of this study is not to make a comparison of the two databases
(ISI and Scopus), but rather apply similar methodology to look closer at Scopus,
examining just how balanced the coverage that it offers is, with the Ulrich directory as
the “gold standard” of reference. Scopus’s coverage with respect to Ulrich is evaluated
by taking a series of variables into account such as journal subject distribution,
geographical distribution, distribution by publishers, the language of publication, and
whether peer-reviewed or not.
F. DE MOYA-ANEGÓN et al.: Coverage analysis of Scopus
Scientometrics 73 (2007)
55
Material and methodology
The two journal collections under comparison are Scopus and the Core version of
Ulrich’s Directory. At present UlrichsWeb.com is the directory covering the greatest
number of journal publications, with over 230,000 titles from approximately 138,000
publishers.
The Serials Analysis System facilitates comparisons between subject categories,
publishers, and the language and country of publication, all done working from a base
collection of journals both in Ulrich’s Universe, which includes all titles (234,173 as of
October 2005) and in Ulrich’s Core, which contains journals of an academic nature and
peer-reviewed (60,859 titles in October 2005). Using the bibliographic information
from each journal, the following aspects can be determined: which journals are peer-
reviewed, their availability in databases, subscription characteristics, inclusion in the
JCR lists, etc. We target two of these areas: the Scopus collection will be analysed and
compared with Ulrich’s Core in relation to the number of subject areas covered, number
of journals in each area (geographic and thematic), and type of review process (peer-
reviewed or not).
A key comparison of ISI databases with those of Ulrich’s Directory was published
in the Homage to Garfield in 2000. Our study differs both in point of reference and in
subject matter. Whereas the previous authors considered only Science and Technology
according to Ulrich CD (B
RAUN et al., 2000), our analysis uses the online version of
Ulrich’s Core (www.ulrichsweb.com) to include all subject categories from the
academic world, not just the basic and experimental sciences of Ulrich’s Science &
Technology (U-S&T). Also, he applied two other conditions concerning journal
selection: titles which do not have an ISSN code, and those containing a book, abstract,
monograph, etc., in the title were not excluded. In this study matching is done with all
the ISSN’s of both collections.
Assuming that Ulrich’s Periodicals Directory represents the worldwide journal
collection, our aim is to determine to what extent it resembles the Scopus collection.
Two similarity measures are used to establish correlations between the distributions of
journals for the two separate collections. On the one hand, the determination coefficient
(R squared value) is calculated using the tendency lines of the distribution graphs,
which indicate the degree to which estimated and real values correspond. Then, in order
to corroborate this similarity, the Pearson correlation coefficient is applied, as previous
authors have done (B
RAUN et al., 2000).
F. DE MOYA-ANEGÓN et al.: Coverage analysis of Scopus
56 Scientometrics 73 (2007)
Results
Scopus has a total of 16,658 titles in June 2005, of which just 13,832 initially
correspond to the ISSN’s in Ulrich’s Directory. This mismatch is explained by the data
shown in Table 1.
Table 1. Scopus collection characteristics
ISSN Scopus Titles %
Matching Ulrich’s Universe 12996 78.02
Matching Ulrich’s Core 11529 69.21
Title History ISSN 433 2.6
Incomplete Serial Record 1 0.01
Invalid ISSN 1 0.01
ISSN Non-Match 214 1.28
Not active 1 0.01
Mergers and Acquisitions 37 0.22
Ceased 146 0.88
Suspended 9 0.05
ISSN SCOPUS in ULRICHSWEB 13838 83.07
Not ISSN/ISBN 1386 8.32
ISBN 1232 7.4
Repetition 202 1.21
Total Scopus 16658 100
Table 2 shows a general comparison of the main areas recorded in the Universe
Version of Ulrich’s Web; it includes all titles, as does the Core version (though the
latter includes only peer-reviewed titles).
Table 2. Scopus vs. Ulrich’s Universe / Ulrich’s Core comparison
Ulrich’s Web SCOPUS ULRICH Percentage Ulrich’s Universe
titles library does
not hold
Titles Universe 12,996 234,173 5.55% 221,177
Core 11,529 60,859 18.94% 49,330
ISSN ULRICH (List Matches) Universe 12,996 197,076 6.59% 184,080
Core 11,529 56,527 20.40% 44,998
Refeered Universe 9,615 29,572 32.51% 19,957
Core 9,615 29,572 32.51% 19,957
Abstracted/Indexed Universe 12,597 73,708 17.09% 61,111
Core 11,273 41,049 27.48% 29,756
Electronic Journal Universe 10,189 56,215 18.13% 46,026
Core 9,535 29,309 32.53% 19,774
Magazines for Libraries Universe 2,712 11,646 23.29% 8,934
Core 2,601 7,458 34.88% 4,857
F. DE MOYA-ANEGÓN et al.: Coverage analysis of Scopus
Scientometrics 73 (2007)
57
Scopus matches 5.55% of the journal titles in relation to Ulrich’s Universe version.
These represent, in turn, 6.59% of titles that match and of those being analysed; 32.51%
of these are peer-reviewed, 17.09% are indexed in databases, and 18.13% are electronic
journals.
In relation to the Core version, Scopus represents 18.94% of the Ulrich collection
and comparisons are carried out starting from 20.40% of the matching journals. Out of
this percentage, 32.51% are peer-reviewed, 27.48% are available on data bases, 32.53%
are electronic journals. Altogether, 62.22% are listed in the JCR. These are the values
that will be used to establish the threshold; the variables studied will be evaluated as
above or below this average value, that is, over or under-represented.
Subject distribution
As a result of the matching process with the Scopus collection, subject distribution
in Ulrich’s Core was seen to divide knowledge into 151 categories, while Scopus has
titles in 120 of them. This means that its subject coverage compared to the worldwide
point of reference is 79.47%, taking into account the 2,820 titles made reference to
above (Table 1). Figure 1 shows the subject distribution of the two journal collections
from a dual perspective.
Figure 1. Journal distribution by subject areas – Ulrich’s Core
