When Does Retargeting Work? Information Specificity in Online Advertising

TL;DR: In this article, the authors use data from a field experiment conducted by an online travel firm to examine whether dynamic retargeted ads are more effective than simply showing generic brand ads.

Abstract: Firms can now offer personalized recommendations to consumers who return to their website, using consumers' previous browsing history on that website. In addition, online advertising has greatly improved in its use of external browsing data to target Internet ads. Dynamic retargeting integrates these two advances by using information from the browsing history on the firm's website to improve advertising content on external websites. When surfing the Internet, consumers who previously viewed products on the firm's website are shown ads with images of those same products. To examine whether this is more effective than simply showing generic brand ads, the authors use data from a field experiment conducted by an online travel firm. Surprisingly, the data suggest that dynamic retargeted ads are, on average, less effective than their generic equivalents. However, when consumers exhibit browsing behavior that suggests their product preferences have evolved (e.g., visiting review websites), dynamic retargeted ad...

Summary

1 Introduction

• Innovations in the parsing and processing of individual-level browsing data enable firms to serve product recommendations in real time to consumers who return to their website.
• These personalized ‘recommendation systems’ often highlight the specific products that the consumer was browsing before they left the website, and may increase sales (Linden et al., 2003; Dias et al., 2008).
• Advertisers currently do not know whether this technique is indeed effective.
• The authors suggest that the effectiveness of a retargeted ad depends on whether the concreteness of its message matches how narrowly consumers construe their preferences (Trope et al., 2007; Lee et al., 2010).
• In the lab, the authors rule out alternative explanations such as privacy, reactance, social validation and sample selection that could also potentially explain why after visiting a travel review site consumers react more positively to a specific than to a generic ad.

2 Relationship to Prior Literature

• Table 1 summarizes how their research relates to previous work on personalized recom- mendations, tailored communications and targeting.
• Research on personalized recommendations on a firm’s website has focused on both documenting their effectiveness (Dias et al., 2008) and on suggesting ways of improving their effectiveness (Linden et al., 2003).
• They do not reach consumers who do not return to the site.
• Similarly, the literature on tailoring communications consistently finds that tailoring improves the performance of communications.
• Third, the authors study the tailoring of ad content, not simply selecting who sees ads based on prior browsing behavior.

3.1 Retargeting

• Ad networks aggregate advertising space across multiple publishers of web content and then sell this space to advertisers.
• For each product page the consumer views, a pixel tag, that is a 1×1 image, is downloaded automatically, recording the fact that that consumer was looking at a specific product.
• The advertising network uses the cookie to identify that the consumer has previously visited the website of the focal firm.
• Dynamic retargeted ads use standardized designs where a predefined space is subdivided into multiple areas for images of specific products.
• The consumer purchases from the focal firm’s website.

3.2 Data

• The authors use data on a travel website that sold hotel stays and hotel vacation packages to consumers.
• The consumer was randomly exposed to a generic or a dynamic retargeted ad when they subsequently visited an external website where the ad network showed ads on behalf of the firm.
• As the firm was a major advertiser on the main travel review sites, it believes that most visitors to a travel review site would have been exposed to its advertising.
• Also, consumers who had viewed a specific type of ad content on a particular day were not more likely to receive either a generic or a dynamic ad on that day (viewed travel website p=0.19, viewed news website p=0.21).
• In their data it would appear that for individuals who have previously visited the firm’s website, contextual ads are extremely successful and that retargeted ads are unsuccessful.

4.1 Generic Retargeting Performs Better on Average

• The authors first explore whether generic and dynamic retargeted ads differ in their effectiveness in converting a consumer to purchase.
• The baseline hazard, h0(t), and the vector of covariates, (Xit).
• Β3 controls for whether the person had seen another form of behavioral targeted ad and β4 measures response to a contextual targeted ad.
• The estimates for the controls do not have a clear causal interpretation.
• Column (6) measures the same-day effect of advertising on purchasing, while also controlling for the effect of a one-day lag of exposure to retargeted ads and the lagged values of each of their cumulative counts of ad exposure.

4.2.1 Theoretical Grounding

• The result that on average dynamically retargeted ads underperform is surprising.
• They may know, for example, they are looking for a relaxing vacation but not whether they prefer a large hotel with a large pool or a small and more intimate hotel that may not feature a pool.
• They also learn the weights to place on different attributes (Hoeffler and Ariely, 1999).
• The authors propose that ads that convey information on high-level characteristics are more effective when consumers have a broad idea of what they want.
• Such consumers are more likely to narrowly construe their preferences and will explore specific choices instead of focusing on their higher level goal.

4.2.2 Empirical Results on Browsing

• Empirically identifying an indicator of whether a consumer has yet developed narrow preferences and is more positively disposed towards a dynamic ad is challenging.
• The authors recognize that a review site visit may be a manifestation of many other different phenomena and explore alternate explanations in detail in their empirical analysis.
• Such misclassification would introduce classification error into their specification.
• This result means that the effectiveness of the dynamic retargeted ad improves after someone has visited a review site.
• For comparison, in Column (2) of Table 6 the authors report the results for the entire sample.

4.2.3 Additional Evidence of Mechanism

• So far, their robustness checks rule out selection or changes in the environment as alternative explanations for their result.
• If the authors find that under higher involvement, the appeal of a dynamic ad to a consumer who has narrowly construed preferences increases further, then this is indirect evidence that the effect they document is driven by the changing effectiveness of advertising content rather than external factors.
• Since in their data the authors observe whether a consumer was exposed to an ad by the travel firm on a travel content site, they use this as an indicator for browsing a travel content site and the consumer being involved in the category.
• Therefore their estimates should be considered as reflecting and not controlling for this change in competition.
• The increasing size of the coefficient RetargetedAd×DynamicAdContent× BrowsingTravelthatDay after a consumer visits a review site suggests that by contrast dynamic retargeted ads perform relatively better after a consumer visits a travel review site and they are browsing the category that day.

5 Confirming the Results in the Lab

• The authors aim to show that the interpretation of their results hold in a controlled lab environment.
• They are then asked to imagine that on the website of a travel company they broadly looked at hotels in many different regions.
• Including these variables in a regression model does not change the main effect of interest (Column (2)).
• The experiment therefore likewise provides evidence that social validation through or access to quality information on a review site is not the primary driver of their results.
• The results confirm that whether a consumer has narrowly construed preferences is an important determinant for the effectiveness of generic versus dynamic retargeted ads.

6 Conclusion

• The digital revolution has seen advances in the use of data on browsing behavior both inside and outside a firm’s website to improve its marketing appeals.
• External browsing data has allowed firms to target their ads better to consumers who fit a particular profile, such as people who have recently been browsing travel websites.
• There is, however, little evidence to show whether tailoring advertising content to an individual’s observed preferences is effective.
• The authors build on a consumer behavior literature which suggests that such a specific emphasis on product features will be most effective when a consumer has established narrowly construed preferences.
• Second, the authors show that the effectiveness of dynamic retargeted ads changes as consumers define their product preferences better and when browsing related content online.

Figures

Table 8: Lab Study One - Travel

Table 1: Previous Literature

Figure 1: Dynamic Retargeting Examples

Figure 2: Conversion Rate with Same-Day Ad Exposure

Table 2: Summary of Online Advertising Methods.

Table 3: Consumers Eligible for Dynamic Retargeting

Figure 4: Comparison of Conversion for Generic vs Dynamic Ad Exposure: Sample Restricted to Those Who Visited A Review Site

Figure 3: Comparison of Conversion for Generic vs Dynamic Ad Exposure

Table 7: Further Evidence of Moderating Effect of Category Activity

Table 4: All Consumers

Figure 5: Comparison of Conversion for Generic vs Dynamic Ad Exposure: Sample Restricted to Consumers Who Browsed a Travel Website and a Review Website

Figure 6: Comparison of Conversion for Generic vs Dynamic Ad exposure: Sample Restricted to Consumers Who Browsed a Travel Website and a Review Website

Full text

When Does Retargeting Work? Information Specificity
in Online Advertising
∗
Anja Lambrecht
†
and Catherine Tucker
‡
May 6, 2013
∗
We thank Havas Digital and particularly Katrin Ribant for access to data from Artemis and Marco
Bertini for facilitating the contact to Havas Digital. We gratefully acknowledge financial support from the
London Business School Centre for Marketing. We thank Kristin Diehl, Anindya Ghose, Avi Goldfarb,
Brett Gordon, Duncan Simester, Catalina Stefanescu, Florian von Wangenheim and Ken Wilbur for their
comments, as well as participants at the 2011 SICS conference and seminar participants at Cass Business
School, ESMT, ESSEC, London Business School and the National University of Singapore.
†
London Business School, London, UK; alambrecht@london.edu.
‡
MIT Sloan School of Management, MIT, Cambridge, MA; cetucker@mit.edu. and NBER
1

When Does Retargeting Work? Information Specificity in Online
Advertising
Abstract
Firms can now serve personalized recommendations to consumers who return to
their website, based on their earlier browsing history on that website. At the same
time, online advertising has greatly advanced in its use of external browsing data across
the web to target internet ads. ‘Dynamic Retargeting’ integrates these two advances,
by using information from earlier browsing on the firm’s website to improve internet
advertising content on external websites. Consumers who previously visited the firm’s
website when surfing the wider web, are shown ads that contain images of products
they have looked at before on the firm’s own website. To examine whether this is more
effective than simply showing generic brand ads, we use data from a field experiment
conducted by an online travel firm. We find, surprisingly, that dynamic retargeted ads
are on average less effective than their generic equivalent. However, when consumers
exhibit browsing behavior such as visiting review websites that suggests their product
preferences have evolved, dynamic retargeted ads no longer underperform.
1

1 Introduction
Innovations in the parsing and processing of individual-level browsing data enable firms to
serve product recommendations in real time to consumers who return to their website. These
personalized ‘recommendation systems’ often highlight the specific products that the con-
sumer was browsing before they left the website, and may increase sales (Linden et al., 2003;
Dias et al., 2008). However, consumers who browse products online often leave the website
without buying and do not return. To reach out to such consumers, dynamic retargeted ads
feature pictures of precisely the product consumers previously browsed.
At first blush, this makes sense: The marketing literature has emphasized that greater
specificity of a firm’s interactions with consumers should increase relevance and consumer
response (Hoffman and Novak, 1996; Komiak and Benbasat, 2006; Dias et al., 2008). Firms
that offer retargeting services point to strong increases in advertising effectiveness. For ex-
ample, Criteo (2010) reports that personalized retargeted ads are six times more effective
than standard banner ads, and four times more effective than retargeting that uses generic
ads. As a result, dynamic retargeting has attracted much enthusiasm among online advertis-
ing practitioners (Hunter, 2010; Hunter et al., 2010; Hargrave, 2011). For example, a single
firm that sells retargeting solutions, the ‘Next Performance’ ad network, reports that it has
served 30 billion retargeted impressions, analyzed 1 billion products for possible inclusion in
a dynamic retargeted ad, and served dynamic retargeted ads to 500 million unique visitors.
1
However, there is little empirical evidence that a personalized product recommendation
is as effective when displayed on external websites, as it is when it is displayed internally
on the firm’s own website. Personalized recommendation systems were designed to sell to
consumers who are engaged enough to return to a firm’s website. Dynamic retargeting, on
the other hand, tries to engage people who have not yet returned to the firm’s website.
1
See http://www.nextperformance.com.

Despite much enthusiasm about dynamic retargeting, advertisers currently do not know
whether this technique is indeed effective. Advertisers also do not know what information
they can use to determine when to show such ads that feature content that is highly specific
to an individual consumer. This research seeks to fill these gaps.
We empirically explore these questions using data from an online field experiment by a
travel company. After consumers visited the travel company’s website and looked at hotels,
an ad network showed banner ads on behalf of the travel company to these consumers when
they subsequently browsed other websites. On these other websites, consumers randomly
either saw an ad that contained an image of the specific hotel the consumer had previously
browsed plus three similar hotels (dynamic retargeting) or an ad that showed a generic brand
ad for the travel firm (generic retargeting). We find that, surprisingly, on average dynamic
retargeting is not effective.
The crucial question for advertisers and ad networks, however, is when dynamic retar-
geting is effective in converting consumers to purchase. We suggest that the effectiveness of
a retargeted ad depends on whether the concreteness of its message matches how narrowly
consumers construe their preferences (Trope et al., 2007; Lee et al., 2010).
Consumers may initially have only a broad idea what they like. Their preferences are
construed at a high level and they focus on higher-level goals. For example, they might want a
‘relaxing vacation’. Over time, consumers shift their focus to specific product attributes and
they develop narrowly construed preferences. For example, they may look for a hotel with
a large swimming pool near the beach. In using the term ’narrowly construed preferences,’
we therefore refer to consumers having a detailed viewpoint on what kind of products they
wish to purchase. We propose that consumers who focus on higher level goals respond better
to advertising messages that address such higher level goals than to messages that display
specific products. Only consumers with narrowly construed preferences are therefore likely
to respond positively to the content of dynamically retargeted ads.
3

We empirically explore whether the effectiveness of a dynamically retargeted ad changes
in parallel with consumers’ browsing behavior which reflects such a shift in goals. We isolate
browsing behavior which may indicate that a consumer has shifted to having narrowly con-
strued preferences and may be more receptive to such highly-specific advertising. We use as
a proxy whether a consumer has visited a travel review site. Wheb searching product-specific
information on a review site, such as TripAdvisor, a consumer is comparing and contrasting
product features and confronting the trade-offs inherent in a product choice. A visit also
provides evidence that consumers are prepared to evaluate products on a detailed level and
indicates that the consumer is thinking deeply about specific products. Therefore we take
a visit to a review site as a potential proxy measure that a consumer now has narrowly
construed preferences.
We find that generic ads are most effective before consumers seek out product quality
information at a review site. Dynamic retargeting becomes relatively more effective only
after consumers have visited a product review site. We find that the greater effectiveness of
retargeting further increases for consumers who are at that time also browsing category-level
content. This is consistent with literature which suggests that the quality of the advertising
message is mediated by consumers’ involvement (Petty et al., 1983).
We acknowledge that visiting a travel review website, as well as being a proxy for having
narrowly construed preferences, could also be a proxy for many other things. In the lab,
we rule out alternative explanations such as privacy, reactance, social validation and sample
selection that could also potentially explain why after visiting a travel review site consumers
react more positively to a specific than to a generic ad.
Therefore, our findings suggest that dynamic retargeting is effective at encouraging con-
sumers to purchase when consumers have visited a review site and are actively browsing
other websites in the category. In all other settings, generic retargeting is more effective.
Our findings about the optimal content of retargeted ads are important given the growth
4

Frequently asked questions

Q1. What are the contributions mentioned in the paper "When does retargeting work? information specificity in online advertising∗" ?

To examine whether this is more effective than simply showing generic brand ads, the authors use data from a field experiment conducted by an online travel firm. However, when consumers exhibit browsing behavior such as visiting review websites that suggests their product preferences have evolved, dynamic retargeted ads no longer underperform. 

Q2. What is the effect of a consumer’s uncertainty about making a category purchase?

As a consumer’s uncertainty about making a category purchase decreases, the psychological distance to the event diminishes (Trope et al., 2007). 

Q3. What is the effect of a dynamic ad on a consumer?

If the authors find that under higher involvement, the appeal of a dynamic ad to a consumer who has narrowly construed preferences increases further, then this is indirect evidence that the effect the authors document is driven by the changing effectiveness of advertising content rather than external factors. 

Q4. How are the results robust to parametric specifications?

Their results are robust to parametric specifications where the baseline hazard is modeled using the Weibull and Exponential distribution. 

Q5. What is the way to use dynamic retargeting?

It suggests that dynamic retargeting is best employed when managers also have access to external browsing data that would help them identify if preferences have evolved and so when dynamic retargeting will be effective.