In this paper we study bid optimisation for real-time bidding (RTB) based display advertising. RTB allows advertisers to bid on a display ad impression in real time when it is being generated. It goes beyond contextual advertising by motivating the bidding focused on user data and it is different from the sponsored search auction where the bid price is associated with keywords. For the demand side, a fundamental technical challenge is to automate the bidding process based on the budget, the campaign objective and various information gathered in runtime and in history. In this paper, the programmatic bidding is cast as a functional optimisation problem. Under certain dependency assumptions, we derive simple bidding functions that can be calculated in real time; our finding shows that the optimal bid has a non-linear relationship with the impression level evaluation such as the click-through rate and the conversion rate, which are estimated in real time from the impression level features. This is different from previous work that is mainly focused on a linear bidding function. Our mathematical derivation suggests that optimal bidding strategies should try to bid more impressions rather than focus on a small set of high valued impressions because according to the current RTB market data, compared to the higher evaluated impressions, the lower evaluated ones are more cost effective and the chances of winning them are relatively higher. Aside from the theoretical insights, offline experiments on a real dataset and online experiments on a production RTB system verify the effectiveness of our proposed optimal bidding strategies and the functional optimisation framework.

/pdf/optimal-real-time-bidding-for-display-advertising-3briokkrqa.pdf

Optimal real-time bidding for display advertising

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a strategic response with respect to other advertisers bidding in the market. In this paper, we formulate bidding optimization with multi-agent reinforcement learning. To deal with a large number of advertisers, we propose a clustering method and assign each cluster with a strategic bidding agent. A practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed and implemented to balance the tradeoff between the competition and cooperation among advertisers. The empirical study on our industry-scaled real-world data has demonstrated the effectiveness of our methods. Our results show cluster-based bidding would largely outperform single-agent and bandit approaches, and the coordinated bidding achieves better overall objectives than purely self-interested bidding agents.

Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

The most significant progress in recent years in online display advertising is what is known as the Real-Time Bidding (RTB) mechanism to buy and sell ads. RTB essentially facilitates buying an individual ad impression in real time while it is still being generated from a user's visit. RTB not only scales up the buying process by aggregating a large amount of available inventories across publishers but, most importantly, enables direct targeting of individual users. As such, RTB has fundamentally changed the landscape of digital marketing. Scientifically, the demand for automation, integration and optimisation in RTB also brings new research opportunities in information retrieval, data mining, machine learning and other related fields. In this monograph, an overview is given of the fundamental infrastructure, algorithms, and technical solutions of this new frontier of computational advertising. The covered topics include user response prediction, bid landscape forecasting, bidding algorithms, revenue optimisation, statistical arbitrage, dynamic pricing, and ad fraud detection.

Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting

Display advertising is a $50 billion industry in which advertisers’ (e.g., P&G, Geico) demand for impressions is matched to publishers’ (e.g., Facebook, Wall Street Journal) supply of them. An idea...

Online Display Advertising Markets: A Literature Review and Future Directions

Taobao, as the largest online retail platform in the world, provides billions of online display advertising impressions for millions of advertisers every day. For commercial purposes, the advertisers bid for specific spots and target crowds to compete for business traffic. The platform chooses the most suitable ads to display in tens of milliseconds. Common pricing methods include cost per mille (CPM) and cost per click (CPC). Traditional advertising systems target certain traits of users and ad placements with fixed bids, essentially regarded as coarse-grained matching of bid and traffic quality. However, the fixed bids set by the advertisers competing for different quality requests cannot fully optimize the advertisers' key requirements. Moreover, the platform has to be responsible for the business revenue and user experience. Thus, we proposed a bid optimizing strategy called optimized cost per click (OCPC) which automatically adjusts the bid to achieve finer matching of bid and traffic quality of page view (PV) request granularity. Our approach optimizes advertisers' demands, platform business revenue and user experience and as a whole improves traffic allocation efficiency. We have validated our approach in Taobao display advertising system in production. The online A/B test shows our algorithm yields substantially better results than previous fixed bid manner.

Optimized Cost per Click in Taobao Display Advertising

In this paper, we report the first empirical study and live test of the reserve price optimisation problem in the context of Real-Time Bidding (RTB) display advertising from an operational environment. A reserve price is the minimum that the auctioneer would accept from bidders in auctions, and in a second price auction it could potentially uplift the auctioneer's revenue by charging winners the reserve price instead of the second highest bids. As such it has been used for sponsored search and been well studied in that context. However, comparing with sponsored search and contextual advertising, this problem in the RTB context is less understood yet more critical for publishers because 1) bidders have to submit a bid for each individual impression, which mostly is associated with user data that is subject to change over time. This, coupled with practical constraints such as the budget, campaigns' life time, etc. makes the theoretical result from optimal auction theory not necessarily applicable and a further empirical study is required to confirm its optimality from the real-world system; 2) in RTB an advertiser is facing nearly unlimited supply and the auction is almost done in "last second", which encourages spending less on the high cost ad placements. This could imply the loss of bid volume over time if a correct reserve price is not in place. In this paper we empirically examine several commonly adopted algorithms for setting up a reserve price. We report our results of a large scale online experiment in a production platform. The results suggest the our proposed game theory based OneShot algorithm performed the best and the superiority is significant in most cases.

http://yuan-shuai.info/paper/KDD-2014-empirical-study-reserve-price.pdf

Peter Mason

Papers

An empirical study of reserve price optimisation in real-time bidding