www.business.unsw.edu.au
School of Economics
Australian School of Business
UNSW Sydney NSW 2052 Australia
http://www.economics.unsw.edu.au
ISSN 1323-8949
ISBN 978 0 7334 2783-1
The views expressed in this paper are those of the authors and do not necessarily reflect those of the
School of Economic at UNSW.
Engineering Trust - Reciprocity in the Production of
Reputation Information
Gary Bolton, Ben Greiner, and Axel Ockenfels
School of Economics Discussion Paper: 2009/02
ENGINEERING TRUST
- RECIPROCITY I N T HE PROD UCTION O F RE PUTAT ION IN FO R MATI ON -
GARY BOLTON, BEN GREINER, AND AXEL OCKENFELS
5 March 2009
Abstract. Reciprocal feedback distorts the production and content of reputation
information, hampering trust and trade efficiency. Data from eBay and other sources
combined with laboratory data provide a robust picture of how reciprocity can be
guided by changes in the way feedback information flows through the system,
leading to more accurate reputation information, more trust and more efficient trade.
Keywords: market design, reputation, trust, reciprocity, eBay
JEL classification: C73, C9, D02, L14
Financial support from the German Science Foundation (DFG) and U.S. National Science Foundation (NSF) is
gratefully acknowledged. We thank Brian Burke, Debbie Hofmeyr, Leland Peterson and the rest of eBay‟s Trust&Safety
team for their cooperation in this research project and for sharing data with us, and Ido Erev, Al Roth and seminar
participants at Berlin, Copenhagen, Graz, Harvard, Indiana, Max Planck Jena, Michigan, Royal Holloway London,
Nürnberg, Santa Barbara, and Sydney for helpful comments. We are indebted to Felix Lamouroux, Karin Ruetz, and
Dietmar Ilsen for excellent research assistance and help with the collection of data.
Bolton. Pennsylvania State University, Smeal College of Business, University Park, PA 16802, Tel: 1 (814) 865 0611, Fax:
1 (814) 865 6284, e-mail: gbolton at psu.edu.
Greiner. University of New South Wales, School of Economics, Sydney, NSW 2052, Australia, Tel: +61 2 9385 9701,
Fax: +61 2 9313 6337, e-mail: bgreiner at unsw.edu.au.
Ockenfels. University of Cologne, Department of Economics, Albertus-Magnus-Platz, D-50923 Köln, Germany, Tel:
+49/221/470-5761, Fax: +49/221/470-5068, e-mail: ockenfels at uni-koeln.de.
1
I. Introduction
This paper reports on the repair of an Internet market trust mechanism. While all markets
require some minimum amount of trust (Akerlof, 1970), it is a particular challenge for Internet
markets, where trading is typically anonymous, geographically dispersed, and executed sequentially.
To incentivize trustworthiness, Internet markets commonly employ reputation-based „feedback
systems‟ that enable traders to publicly post information about past transaction partners. Online
markets with this kind of feedback system include eBay.com, Amazon.com, and RentACoder.com,
among many others. For these markets, feedback systems with their large databases of transaction
histories are a core asset, crucial for market efficiency and user loyalty.
Engineering studies examine institutional design problems at a fine-grained level. They
constitute a unique kind of testing ground for existing concepts and for identifying new questions
for economic theory.
1
In the present case, economic theory implies that a reputation system that
elicits accurate and complete feedback information can promote trust and cooperation among
selfish traders even in such adverse environments as online market platforms.
2
So there is theoretical
reason to believe that a properly designed feedback system can effectively facilitate trade. At the
same time, the nature of the problem takes us further down the causation chain than received theory
presently goes, to gaming in the production of reputation information. In essence, reputation builders
retaliate in-kind for a negative review, thereby inhibiting the provision of negative reviews in the first
place. The resulting bias in reputation information then works its way up the chain, ultimately
diminishing market efficiency. Other reputation-based systems are open to similar retaliation (ex.,
financial rating services, employee job assessments, word-of-mouth about colleagues), so the
phenomenon likely has a life beyond Internet markets.
Below we present new data from the eBay marketplace exhibiting a strong and general reciprocal
pattern in the content, timing and quantity of reputation information (Section II). It turns out that
the institutional trigger for this behavior is the timing and posting rules governing feedback giving.
The natural approach to fixing the problem then is to change these rules to diminish reciprocal
behavior. Doing so involved two large uncertainties. First, it was not clear how responsive the
1
Roth (2002) argues the need for a literature “to further the design and maintenance of markets and other economic
institutions [p. 1341]” and provides examples of how this literature can help shape new questions in economic theory.
To date, the market design literature has focused mostly on allocation mechanisms such as auctions and matching. Roth
(2008) reviews the literature on matching markets, Milgrom (2004) the literature on auction markets, and Ockenfels,
Reiley, and Sadrieh (2006) the literature specific to Internet auctions.
2
See, for example, Wilson (1985) and, for the literature specific to feedback systems, Dellarocas (2006).
2
system would be: In order to influence trade, the new system need evoke subtle, strategically
motivated changes in the behavior of the traders, at multiple points, as the information flows
through the market. Second, changing the feedback rules risks undesirable side effects, as reciprocal
feedback has positive, as well as negative consequences. Most critically, reciprocity appears
important to getting (legitimately) satisfactory trades reported; eliminating this kind of reciprocity
might lead to a system that over reports, rather than under reports, negative outcomes.
With these considerations in mind, our study examines two alternative proposals (also Section
II).
3
Analyzing data from other Internet markets that have feedback systems with features similar to
those proposed suffices to answer some of our questions (Section III). But not all of them. There
are behavioral and institutional differences across the markets we examine and this leaves substantial
ambiguity; one proposal, in particular, has major features not shared with any existing market. Also,
we lack field data on the underlying cost and preference parameters in the markets, and so cannot
easily measure how feedback systems affect market efficiency. To narrow the uncertainty, we
complement the field data with a test bed experiment crafted to capture the theoretically relevant
aspects of behavior and institutional changes (Section IV).
4
In combination, the field and lab data provide a robust picture of how reciprocity can be guided
through the design of information channels. We will see that more accurate information sets off an
endogenous shift in the market, leading to greater trust and trustworthiness, and more efficient
trade. Our analysis guided eBay in its decision to change the reputation system. We present
preliminary data on how the new field system performs (Section V). We conclude with lessons for
market design and future theory building (Section VI).
II. The feedback problem and two proposals to fix it
In this section we first review eBay‟s conventional feedback system (Subsection II.1). We then
examine evidence, from new data as well as from the work of other researchers, for a reciprocal
pattern in feedback giving and for the role of the rules that govern feedback giving (Subsection II.2).
3
A number of others proposals were considered but discarded relatively quickly in favor of the two discussed here.
4
Test bed experiments to get insight into how a market redesign will work has been done in relation to allocation
mechanisms; for example, Grether, Isaac, and Plott (1981), Kagel and Roth (2000), Chen (2005), Kwasnica, Ledyard,
Porter, and DeMartini (2005), Chen and Sönmez (2006) and Brunner, Goeree, Holt, and Ledyard (forthcoming).
3
An important point will be that reciprocal behavior has good as well as bad consequences for the
system. We then discuss two proposals put forward to mitigate the problem (Subsection II.3).
5
II.1 EBay’s conventional feedback system
EBay facilitates trade in the form of auctions and posted offers in over thirty countries.
6
After
each eBay transaction, both the buyer and the seller are invited to give feedback on each other. Until
spring 2007, when eBay changed the system, only “conventional” feedback could be left. In the
conventional feedback system, a trader can rate a transaction positive, neutral, or negative (along
with a text comment). Submitted feedback is immediately posted and available to all traders.
Conventional feedback ratings can be removed from the site only by court ruling, or if the buyer did
not pay, or if both transaction partners mutually agree to withdrawal.
7
The most common summary measure of an eBay trader‟s feedback history is the feedback score,
equal to the difference between the number of positive and negative feedbacks from unique eBay
traders (neutral scores are ignored). Each trader‟s feedback score is provided on the site. An
important advantage of the feedback score is that it incorporates a reliability measure (experience) in
the measure of trustworthiness. The feedback score is also the most commonly used measure of
feedback history in research analyses of eBay data.
8
II.2 Reciprocal feedback
Feedback information is largely a public good, helping other traders to manage the risks
involved in trusting unknown transaction partners. Yet our data finds that about 70% of the traders
5
That said, many (but not all) studies find that feedback has positive value for traders as indicated by positive
correlations between the feedback score of a seller and the revenue and the probability of sale. See, for example, Bajari
and Hortaçsu (2003, 2004), Ba and Pavlou (2002), Cabral and Hortaçsu (forthcoming), Dellarocas (2004), Dewan and
Hsu (2001), Eaton (2007), Ederington and Dewally (2006), Houser and Wooders (2005), Jin and Kato (forthcoming),
Kalyanam and McIntyre (2001), Livingston (2005), Livingston and Evans (2004), Lucking-Reiley, Bryan, Prasad, and
Reeves (2007), McDonald and Slawson (2002), Melnik and Alm (2002), Ockenfels (2003), Resnick and Zeckhauser
(2002), and Resnick, Zeckhauser, Swanson, and Lockwood (2006). See Ba and Pavlou (2002), Bolton, Katok, and
Ockenfels (2004, 2005), and Bolton and Ockenfels (forthcoming) for laboratory evidence.
6
In 2006, 82 million users bought or sold $52 billion in goods on eBay platforms.
7
EBay‟s old feedback system was the product of an 11 year evolutionary process. In its first version, introduced in 1996,
feedback was not bound to mutual transactions: every community member could give an opinion about every other
community member. In 1999/2000 the ability to submit non-transaction related feedback was removed. The percentage
of positive feedback was introduced in 2003, and in 2004 the procedure of mutual feedback withdrawal was added. Since
2005, feedback submitted by eBay users leaving the platform shortly thereafter or not participating in „issue resolution
processes‟ is made ineffective, and members who want to leave neutral or negative feedback must go through a tutorial
before being able to do so. Since spring 2007 a new system was introduced, as described in Section V. In 2008, again
new features were implemented, which are analyzed in Bolton, Greiner and Ockenfels (2009).
8
Another common measure is the „percentage positive‟ equal to the share of positive and negative feedbacks that is
positive. For our data, which measure is used seems to make little difference; we mostly report results using the feedback
score.
