Item-Based Collaborative Filtering Recommendation
Algorithms
Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl
f
sarwar, karypis, konstan, riedl
g
@cs.umn.edu
GroupLens Research Group/Army HPC Research Center
Department of Computer Science and Engineering
University of Minnesota, Minneapolis, MN 55455
ABSTRACT
Recommender systems apply knowledge discovery techniques
to the problem of making personalized recommendations for
information, pro ducts or services during a liveinteraction.
These systems, esp ecially the k-nearest neighbor collabora-
tive ltering based ones, are achieving widespread success on
the Web. The tremendous growth in the amountof avail-
able information and the number of visitors to Web sites in
recentyears p oses some key challenges for recommender sys-
tems. These are: pro ducing high quality recommendations,
performing many recommendations p er second for millions
of users and items and achieving high coverage in the face of
data sparsity. In traditional collaborative ltering systems
the amountof work increases with the number of partici-
pants in the system. New recommender system technologies
are needed that can quickly produce high quality recom-
mendations, even for very large-scale problems. To address
these issues we have explored item-based collaborative l-
tering techniques. Item-based techniques rst analyze the
user-item matrix to identify relationships b etween dierent
items, and then use these relationships to indirectly compute
recommendations for users.
In this pap er we analyze dierent item-based recommen-
dation generation algorithms. Welook into dierenttech-
niques for computing item-item similarities (e.g., item-item
correlation vs. cosine similarities between item vectors) and
dierenttechniques for obtaining recommendations from them
(e.g., weighted sum vs. regression model). Finally, weex-
perimentally evaluate our results and compare them to the
basic k-nearest neighbor approach. Our experiments sug-
gest that item-based algorithms provide dramatically better
performance than user-based algorithms, while at the same
time providing better quality than the b est available user-
based algorithms.
1. INTRODUCTION
The amount of information in the world is increasing far
more quickly than our ability to pro cess it. All of us have
known the feeling of b eing overwhelmed by the number of
new bo oks, journal articles, and conference pro ceedings com-
ing out eachyear. Technology has dramatically reduced the
barriers to publishing and distributing information. Now
it is time to create the technologies that can help us sift
Copyright is held by the author/owner.
WWW10, May 1-5, 2001, Hong Kong.
ACM 1-58113-348-0/01/0005.
through all the available information to nd that whichis
most valuable to us.
One of the most promising suchtechnologies is
col labora-
tive ltering
[19, 27, 14, 16]. Collab orative ltering works by
building a database of preferences for items by users. Anew
user, Neo, is matched against the database to discover
neigh-
bors
, which are other users who have historically had similar
taste to Neo. Items that the neighbors like are then recom-
mended to Neo, as he will probably also like them. Collab-
orative ltering has been very successful in both research
and practice, and in b oth information ltering applications
and E-commerce applications. However, there remain im-
portant research questions in overcoming two fundamental
challenges for collaborative ltering recommender systems.
The rst challenge is to improve the scalability of the col-
laborative ltering algorithms. These algorithms are able to
search tens of thousands of p otential neighbors in real-time,
but the demands of modern systems are to searchtensof
millions of p otential neighbors. Further, existing algorithms
have p erformance problems with individual users for whom
the site has large amounts of information. For instance,
if a site is using browsing patterns as indications of con-
tent preference, it mayhave thousands of data p oints for its
most frequent visitors. These \long user rows" slow down
the number of neighb ors that can b e searched p er second,
further reducing scalability.
The second challenge is to improve the quality of the rec-
ommendations for the users. Users need recommendations
they can trust to help them nd items they will like. Users
will "vote with their feet" by refusing to use recommender
systems that are not consistently accurate for them.
In some ways these twochallenges are in conict, since the
less time an algorithm sp ends searching for neighbors, the
more scalable it will b e, and the worse its quality. For this
reason, it is important to treat the two challenges simul-
taneously so the solutions discovered are b oth useful and
practical.
In this pap er, we address these issues of recommender
systems by applying a dierent approach{item-based algo-
rithm. The b ottleneckinconventional collab orative lter-
ing algorithms is the search for neighbors among a large
user p opulation of p otential neighbors [12]. Item-based al-
gorithms avoid this b ottleneckby exploring the relationships
between items rst, rather than the relationships between
users. Recommendations for users are computed by nding
items that are similar to other items the user has liked. Be-
cause the relationships between items are relatively static,
285
item-based algorithms may b e able to provide the same qual-
ity as the user-based algorithms with less online computa-
tion.
1.1 Related Work
In this section we briey present some of the researchlit-
erature related to collab orative ltering, recommender sys-
tems, data mining and p ersonalization.
Tapestry [10] is one of the earliest implementations of col-
laborative ltering-based recommender systems. This sys-
tem relied on the explicit opinions of people from a close-knit
community,such as an oÆce workgroup. However, recom-
mender system for large communities cannot depend on each
person knowing the others. Later, several ratings-based au-
tomated recommender systems were developed. The Grou-
pLens research system [19, 16] provides a pseudonymous
collaborative ltering solution for Usenet news and movies.
Ringo [27] and Video Recommender [14] are email and web-
based systems that generate recommendations on music and
movies, resp ectively. A sp ecial issue of Communications of
the ACM [20] presents a number of dierent recommender
systems.
Other technologies have also been applied to recommender
systems, including Bayesian networks, clustering, and Hort-
ing. Bayesian networks create a mo del based on a training
set with a decision tree at each no de and edges represent-
ing user information. The mo del can be built o-line over a
matter of hours or days. The resulting model is very small,
very fast, and essentially as accurate as nearest neighbor
methods [6]. Bayesian networks mayprove practical for en-
vironments in whichknowledge of user preferences changes
slowly with resp ect to the time needed to build the model
but are not suitable for environments in which user prefer-
ence models must b e updated rapidly or frequently.
Clustering techniques work by identifying groups of users
who appear to have similar preferences. Once the clusters
are created, predictions for an individual can b e made byav-
eraging the opinions of the other users in that cluster. Some
clustering techniques represent each user with partial par-
ticipation in several clusters. The prediction is then an aver-
age across the clusters, weighted by degree of participation.
Clustering techniques usually pro duce less-p ersonal recom-
mendations than other metho ds, and in some cases, the clus-
ters haveworse accuracy than nearest neighbor algorithms
[6]. Once the clustering is complete, however, p erformance
can b e very goo d, since the size of the group that must b e
analyzed is much smaller. Clustering techniques can also
be applied as a "rst step" for shrinking the candidate set
in a nearest neighb or algorithm or for distributing nearest-
neighbor computation across several recommender engines.
While dividing the p opulation into clusters may hurt the
accuracy or recommendations to users near the fringes of
their assigned cluster, pre-clustering maybe a worthwhile
trade-o between accuracy and throughput.
Horting is a graph-based technique in which no des are
users, and edges between nodes indicate degree of similarity
between two users [1]. Predictions are produced bywalking
the graph to nearbynodesand combining the opinions of
the nearby users. Horting diers from nearest neighbor as
the graph maybewalked through other users who have not
rated the item in question, thus exploring transitive rela-
tionships that nearest neighbor algorithms do not consider.
In one study using synthetic data, Horting pro duced better
predictions than a nearest neighbor algorithm [1].
Schafer et al., [26] present a detailed taxonomy and exam-
ples of recommender systems used in E-commerce and how
they can provide one-to-one personalization and at the same
can capture customer loyalty. Although these systems have
been successful in the past, their widespread use has exp osed
some of their limitations such as the problems of sparsityin
the data set, problems asso ciated with high dimensionality
and so on. Sparsity problem in recommender system has
been addressed in [23, 11]. The problems asso ciated with
high dimensionality in recommender systems havebeendis-
cussed in [4], and application of dimensionality reduction
techniques to address these issues has b een investigated in
[24].
Our work explores the extentto which item-based recom-
menders, a new class of recommender algorithms, are able
to solve these problems.
1.2 Contributions
This paper has three primary researchcontributions:
1. Analysis of the item-based prediction algorithms and
identication of dierentways to implementits sub-
tasks.
2. Formulation of a precomputed mo del of item similarity
to increase the online scalability of item-based recom-
mendations.
3. An exp erimental comparison of the qualityof several
dierent item-based algorithms to the classic user-based
(nearest neighbor) algorithms.
1.3 Organization
The rest of the pap er is organized as follows. The next
section provides a brief background in collab orative lter-
ing algorithms. We rst formally describ e the collaborative
ltering pro cess and then discuss its twovariants memory-
based and mo del-based approaches. We then present some
challenges asso ciated with the memory-based approach. In
section 3, we present the item-based approach and describ e
dierent sub-tasks of the algorithm in detail. Section 4 de-
scribes our exp erimental work. It provides details of our
data sets, evaluation metrics, metho dology and results of
dierent experiments and discussion of the results. The -
nal section provides some concluding remarks and directions
for future research.
2. COLLABORATIVE FILTERING BASED
RECOMMENDER SYSTEMS
Recommender systems
systems apply data analysis tech-
niques to the problem of helping users nd the items they
would like to purchase at E-Commerce sites by pro ducing
a predicted likeliness score or a list of
top{
N
recommended
items for a given user. Item recommendations can b e made
using dierent methods. Recommendations can be based
on demographics of the users, overall top selling items, or
past buying habit of users as a predictor of future items.
Collaborative Filtering (CF) [19, 27] is the most success-
ful recommendation technique to date. The basic idea of
CF-based algorithms is to provide item recommendations
or predictions based on the opinions of other like-minded
286
users. The opinions of users can be obtained
explicitly
from
the users or by using some
implicit
measures.
2.0.1 Overview of the Collaborative Filtering Pro-
cess
The goal of a collab orative ltering algorithm is to sug-
gest new items or to predict the utility of a certain item for
a particular user based on the user's previous likings and
the opinions of other like-minded users. In a typical CF sce-
nario, there is a list of
m
users
U
=
f
u
1
;u
2
;::: ;u
m
g
and a
list of
n
items
I
=
f
i
1
;i
2
;::: ;i
n
g
. Eachuser
u
i
has a list
of items
I
u
i
, which the user has expressed his/her opinions
about. Opinions can be explicitly given by the user as a
rat-
ing score
, generally within a certain numerical scale, or can
be implicitly derived from purchase records, by analyzing
timing logs, by mining web hyperlinks and so on [28, 16].
Note that
I
u
i
I
and it is possible for
I
u
i
to be a
nul l-set
.
There exists a distinguished user
u
a
2U
called the
active
user
for whom the task of a collaborative ltering algorithm
is to nd an item likeliness that can be of two forms.
Prediction
isanumerical value,
P
a;j
, expressing the
predicted likeliness of item
i
j
62
I
u
a
for the activeuser
u
a
. This predicted value is within the same scale (e.g.,
from 1 to 5) as the opinion values provided by
u
a
.
Recommendation
isalistof
N
items,
I
r
I
, that
the active user will like the most. Note that the recom-
mended list must b e on items not already purchased by
the active user, i.e.,
I
r
\
I
u
a
=. This interface of CF
algorithms is also known as
Top-N recommendation
.
Figure 1 shows the schematic diagram of the collab orative
ltering pro cess. CF algorithms represent the entire
m
n
user-item data as a ratings matrix,
A
. Eachentry
a
i;j
in
A
represents the preference score (ratings) of the
i
th user on
the
j
th item. Each individual ratings is within a numerical
scale and it can as well be 0 indicating that the user has
not yet rated that item. Researchers have devised a num-
ber of collaborative ltering algorithms that can b e divided
into two main categories|
Memory-based (user-based)
and
Model-based (item-based)
algorithms [6]. In this section we
provide a detailed analysis of CF-based recommender sys-
tem algorithms.
Memory-based Collab orative Filtering Algorithms
.
Memory-based algorithms utilize the entire user-item data-
base to generate a prediction. These systems employsta-
tistical techniques to nd a set of users, known as
neigh-
bors
, that have a history of agreeing with the target user
(i.e., they either rate dierent items similarly or they tend
to buy similar set of items). Once a neighborhood of users
is formed, these systems use dierent algorithms to com-
bine the preferences of neighbors to pro duce a prediction or
top-
N
recommendation for the activeuser. The techniques,
also known as
nearest-neighbor
or user-based collaborative
ltering, are more popular and widely used in practice.
Model-based Collab orative Filtering Algorithms
.
Mo-
del-based collab orative ltering algorithms provide item rec-
ommendation by rst developing a mo del of user ratings. Al-
gorithms in this category take a probabilistic approach and
envision the collaborative ltering pro cess as computing the
expected value of a user prediction, given his/her ratings
on other items. The mo del building process is p erformed
by dierent
machine learning
algorithms suchas
Bayesian
network, clustering,
and
rule-based
approaches. The
Bayesian network model [6] formulates a probabilistic mo del
for collab orative ltering problem. Clustering mo del treats
collaborative ltering as a classication problem [2, 6, 29]
and works by clustering similar users in same class and esti-
mating the probability that a particular user is in a partic-
ular class
C
, and from there computes the conditional prob-
ability of ratings. The rule-based approach applies associ-
ation rule discovery algorithms to nd asso ciation b etween
co-purchased items and then generates item recommenda-
tion based on the strength of the association b etween items
[25].
2.0.2 Challenges of User-based Collaborative Filter-
ing Algorithms
User-based collaborative ltering systems havebeenvery
successful in past, but their widespread use has revealed
some potential challenges suchas:
Sparsity.
In practice, many commercial recommender
systems are used to evaluate large item sets (e.g., Ama-
zon.com recommends b o oks and CDnow.com recom-
mends music albums). In these systems, even active
users mayhavepurchased well under 1% of the items
(1% of 2 million b ooks is 20
;
000 bo oks). Accordingly,
a recommender system based on nearest neighbor al-
gorithms may be unable to makeany item recommen-
dations for a particular user. As a result the accuracy
of recommendations maybe poor.
Scalability.
Nearest neighbor algorithms require com-
putation that grows with both the number of users
and the number of items. With millions of users and
items, a typical web-based recommender system run-
ning existing algorithms will suer serious scalability
problems.
The weakness of nearest neighbor algorithm for large,
sparse databases led us to explore alternative recommender
system algorithms. Our rst approach attempted to bridge
the sparsityby incorp orating semi-intelligent ltering agents
into the system [23, 11]. These agents evaluated and rated
each item using syntactic features. By providing a dense rat-
ings set, they helped alleviate coverage and improved qual-
ity. The ltering agent solution, however, did not address
the fundamental problem of p oor relationships among like-
minded but sparse-rating users. To explore that we to ok
an algorithmic approach and used Latent Semantic Index-
ing (LSI) to capture the similaritybetween users and items
in a reduced dimensional space [24, 25]. In this pap er we
look into another technique, the model-based approach, in
addressing these challenges, esp ecially the scalability chal-
lenge. The main idea here is to analyze the user-item repre-
sentation matrix to identify relations between dierent items
and then to use these relations to compute the prediction
score for a given user-item pair. The intuition b ehind this
approach is that a user would be interested in purchasing
items that are similar to the items the user liked earlier
and would tend to avoid items that are similar to the items
the user didn't like earlier. These techniques don't require
to identify the neighborhood of similar users when a rec-
ommendation is requested; as a result they tend to pro-
duce much faster recommendations. Anumber of dierent
287
u
1
u
2
u
a
u
m
.
.
.
.
i
1
i
2
i
j
i
n
. .
. .
Input (ratings table)
Active user
Item for which prediction
is sought
Prediction
Recommendation
CF-Algorithm
P
a,j
(prediction on
item j for the active
user)
{T
i1
, T
i2
, ..., T
iN
} Top-N
list of items for the
active user
Output interface
Figure 1: The Collaborative Filtering Process.
schemes have been prop osed to compute the association b e-
tween items ranging from probabilistic approach [6] to more
traditional item-item correlations [15, 13]. We present a de-
tailed analysis of our approach in the next section.
3. ITEM-BASED COLLABORATIVE FILT-
ERING ALGORITHM
In this section we study a class of item-based recommen-
dation algorithms for producing predictions to users. Unlike
the user-based collab orative ltering algorithm discussed in
Section 2, the item-based approach lo oks into the set of
items the target user has rated and computes how simi-
lar they are to the target item
i
and then selects
k
most
similar items
f
i
1
;i
2
;::: ;i
k
g
. At the same time their cor-
responding similarities
f
s
i
1
;s
i
2
;::: ;s
ik
g
are also computed.
Once the most similar items are found, the prediction is
then computed by taking a weighted average of the target
user's ratings on these similar items. We describ e these two
aspects, namely, the similarity computation and the predic-
tion generation in details here.
3.1 Item Similarity Computation
One critical step in the item-based collab orative ltering
algorithm is to compute the similarity between items and
then to select the most similar items. The basic idea in
similarity computation between two items
i
and
j
is to rst
isolate the users who have rated b oth of these items and then
to apply a similarity computation technique to determine
the similarity
s
i;j
. Figure 2 illustrates this pro cess; here
the matrix rows represent users and the columns represent
items.
There are a number of dierentways to compute the sim-
ilaritybetween items. Here we present three such metho ds.
These are cosine-based similarity, correlation-based similar-
ity and adjusted-cosine similarity.
3.1.1 Cosine-based Similarity
In this case, two items are thoughtofas two vectors in
the
m
dimensional user-space. The similaritybetween them
is measured by computing the cosine of the angle between
these two vectors. Formally, in the
m
n
ratings matrix
in Figure 2, similarity between items
i
and
j
, denoted by
sim
(
i; j
)isgiven by
sim
(
i; j
)=cos(
~
i;
~
j
)=
~
i
~
j
k
~
i
k
2
k
~
j
k
2
where \
" denotes the dot-product of the twovectors.
3.1.2 Correlation-based Similarity
In this case, similaritybetween two items
i
and
j
is mea-
sured by computing the
Pearson-r
correlation
corr
i;j
. To
make the correlation computation accurate we must rst
isolate the co-rated cases (i.e., cases where the users rated
both
i
and
j
) as shown in Figure 2. Let the set of users who
both rated
i
and
j
are denoted by
U
then the correlation
similarity is given by
sim
(
i; j
)=
P
u
2
U
(
R
u;i
R
i
)(
R
u;j
R
j
)
q
P
u
2
U
(
R
u;i
R
i
)
2
q
P
u
2
U
(
R
u;j
R
j
)
2
:
Here
R
u;i
denotes the rating of user
u
on item
i
,
R
i
is the
average rating of the
i
-th item.
3.1.3 Adjusted Cosine Similarity
One fundamental dierence between the similaritycom-
putation in user-based CF and item-based CF is that in case
of user-based CF the similarity is computed along the rows
of the matrix but in case of the item-based CF the simi-
larity is computed along the columns, i.e., each pair in the
co-rated set corresponds to a dierent user (Figure 2). Com-
puting similarity using basic cosine measure in item-based
case has one imp ortant drawback|the dierences in rat-
ing scale between dierent users are not taken into account.
The adjusted cosine similarity osets this drawbackby sub-
tracting the corresp onding user average from each co-rated
pair. Formally, the similaritybetween items
i
and
j
using
this scheme is given by
sim
(
i; j
)=
P
u
2
U
(
R
u;i
R
u
)(
R
u;j
R
u
)
q
P
u
2
U
(
R
u;i
R
u
)
2
q
P
u
2
U
(
R
u;j
R
u
)
2
:
Here
R
u
is the average of the
u
-th user's ratings.
288
1
2
3 i n-1 n
1
2
u
m
m-1
j
R-
R -
R R
R R
R R
Item-item similarity is computed by
looking into co-rated items only. In
case of items i and j the similarity s
i,j
is
computed by looking into them. Note:
each of these co-rated pairs are
obtained from different users, in this
example they come from users 1, u
and m-1.
s
i,j
=?
Figure 2: Isolation of the co-rated items and similarity computation
3.2 Prediction Computation
The most important step in a collab orative ltering sys-
tem is to generate the output interface in terms of prediction.
Once we isolate the set of most similar items based on the
similarity measures, the next step is to lo ok into the tar-
get users ratings and use a technique to obtain predictions.
Here we consider two such techniques.
3.2.1 Weighted Sum
As the name implies, this metho d computes the prediction
on an item
i
for a user
u
by computing the sum of the ratings
given by the user on the items similar to
i
. Each ratings is
weighted by the corresponding similarity
s
i;j
between items
i
and
j
. Formally, using the notion shown in Figure 3 we
can denote the prediction
P
u;i
as
P
u;i
=
P
all similar items, N
(
s
i;N
R
u;N
)
P
all similar items, N
(
j
s
i;N
j
)
Basically, this approach tries to capture how the active
user rates the similar items. The weighted sum is scaled by
the sum of the similarity terms to make sure the prediction
is within the predened range.
3.2.2 Regression
This approach is similar to the weighted sum method but
instead of directly using the ratings of similar items it uses
an approximation of the ratings based on regression model.
In practice, the similarities computed using cosine or cor-
relation measures may be misleading in the sense that two
rating vectors may b e distant (in Euclidean sense) yet may
havevery high similarity. In that case using the raw ratings
of the \so-called" similar item may result in po or prediction.
The basic idea is to use the same formula as the weighted
sum technique, but instead of using the similar item
N
's
\raw" ratings values
R
u;N
's, this mo del uses their approx-
imated values
R
0
u;N
based on a linear regression mo del. If
we denote the respectivevectors of the target item
i
and the
similar item
N
by
R
i
and
R
N
the linear regression model
can be expressed as
R
0
N
=
R
i
+
+
The regression model parameters
and
are determined
by going over b oth of the rating vectors.
is the error of
the regression mo del.
3.3 Performance Implications
The largest E-Commerce sites op erate at a scale that
stresses the direct implementation of collab orative ltering.
In neighborho od-based CF systems, the neighb orhoo d for-
mation process, esp ecially the user-user similarity computa-
tion step turns out to be the performance b ottleneck, which
in turn can make the whole pro cess unsuitable for real-time
recommendation generation. One way of ensuring high scal-
ability is to use a mo del-based approach. Model-based sys-
tems have the p otential to contribute to recommender sys-
tems to operate at a high scale. The main idea here to iso-
late the neighborho od generation and prediction generation
steps.
In this pap er, we present a mo del-based approach to pre-
compute item-item similarity scores. The similarity compu-
tation scheme is still correlation-based but the computation
is performed on the item space. In a typical E-Commerce
scenario, we usually have a set of item that is static com-
pared to the number of users that changes most often. The
static nature of items leads us to the idea of precomput-
ing the item similarities. One p ossible way of precomputing
the item similarities is to compute all-to-all similarityand
then p erforming a quick table lo ok-up to retrieve the re-
quired similarityvalues. This method, although saves time,
requires an
O
(
n
2
) space for
n
items.
The fact that we only need a small fraction of similar items
to compute predictions leads us to an alternate model-based
scheme. In this scheme, we retain only a small number of
similar items. For each item
j
we compute the
k
most sim-
ilar items, where
k
n
and record these item numbers
and their similarities with
j
. Weterm
k
as the
model size
.
Based on this mo del building step, our prediction genera-
tion algorithm works as follows. For generating predictions
for a user
u
on item
i
, our algorithm rst retrieves the pre-
computed
k
most similar items corresp onding to the target
item
i
. Then it lo oks howmany of those
k
items were pur-
chased by the user
u
, based on this intersection then the
289
