Predicting user responses, such as click-through rate and conversion rate, are critical in many web applications including web search, personalised recommendation, and online advertising. Different from continuous raw features that we usually found in the image and audio domains, the input features in web space are always of multi-field and are mostly discrete and categorical while their dependencies are little known. Major user response prediction models have to either limit themselves to linear models or require manually building up high-order combination features. The former loses the ability of exploring feature interactions, while the latter results in a heavy computation in the large feature space. To tackle the issue, we propose two novel models using deep neural networks (DNNs) to automatically learn effective patterns from categorical feature interactions and make predictions of users’ ad clicks. To get our DNNs efficiently work, we propose to leverage three feature transformation methods, i.e., factorisation machines (FMs), restricted Boltzmann machines (RBMs) and denoising auto-encoders (DAEs). This paper presents the structure of our models and their efficient training algorithms. The large-scale experiments with real-world data demonstrate that our methods work better than major state-of-the-art models.

Deep Learning over Multi-field Categorical Data

In this paper we study bid optimisation for real-time bidding (RTB) based display advertising. RTB allows advertisers to bid on a display ad impression in real time when it is being generated. It goes beyond contextual advertising by motivating the bidding focused on user data and it is different from the sponsored search auction where the bid price is associated with keywords. For the demand side, a fundamental technical challenge is to automate the bidding process based on the budget, the campaign objective and various information gathered in runtime and in history. In this paper, the programmatic bidding is cast as a functional optimisation problem. Under certain dependency assumptions, we derive simple bidding functions that can be calculated in real time; our finding shows that the optimal bid has a non-linear relationship with the impression level evaluation such as the click-through rate and the conversion rate, which are estimated in real time from the impression level features. This is different from previous work that is mainly focused on a linear bidding function. Our mathematical derivation suggests that optimal bidding strategies should try to bid more impressions rather than focus on a small set of high valued impressions because according to the current RTB market data, compared to the higher evaluated impressions, the lower evaluated ones are more cost effective and the chances of winning them are relatively higher. Aside from the theoretical insights, offline experiments on a real dataset and online experiments on a production RTB system verify the effectiveness of our proposed optimal bidding strategies and the functional optimisation framework.

/pdf/optimal-real-time-bidding-for-display-advertising-3briokkrqa.pdf

Optimal real-time bidding for display advertising

© Springer International Publishing Switzerland 2016. Predicting user responses, such as click-through rate and conversion rate, are critical in many web applications including web search, personalised recommendation, and online advertising. Different from continuous rawfeatures thatwe usually found in the image and audio domains, the input features in web space are always ofmulti-field and aremostly discrete and categorical while their dependencies are little known.Major user response prediction models have to either limit themselves to linear models or require manually building up high-order combination features. The former loses the ability of exploring feature interactions, while the latter results in a heavy computation in the large feature space. To tackle the issue, we propose two novel models using deep neural networks (DNNs) to automatically learn effective patterns from categorical feature interactions and make predictions of users’ ad clicks. To get our DNNs efficiently work, we propose to leverage three feature transformation methods, i.e., factorisation machines (FMs), restricted Boltzmann machines (RBMs) and denoising auto-encoders (DAEs). This paper presents the structure of our models and their efficient training algorithms. The large-scale experiments with real-world data demonstrate that our methods work better than major state-of-the-art models.

Deep learning over Multi-Field categorical Data - A case study on user response prediction

In recent years, there are a large number of recommendation algorithms proposed in the literature, from traditional collaborative filtering to neural network algorithms. However, the concerns about how to standardize open source implementation of recommendation algorithms continually increase in the research community. 
In the light of this challenge, we propose a unified, comprehensive and efficient recommender system library called RecBole, which provides a unified framework to develop and reproduce recommender systems for research purpose. In this library, we implement 53 recommendation models on 27 benchmark datasets, covering the categories of general recommendation, sequential recommendation, context-aware recommendation and knowledge-based recommendation. We implement the RecBole library based on PyTorch, which is one of the most popular deep learning frameworks. Our library is featured in many aspects, including general and extensible data structures, comprehensive benchmark models and datasets, efficient GPU-accelerated execution, and extensive and standard evaluation protocols. We provide a series of auxiliary functions, tools, and scripts to facilitate the use of this library, such as automatic parameter tuning and break-point resume. Such a framework is useful to standardize the implementation and evaluation of recommender systems. The project and documents are released at this https URL.

RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms

A recent “third wave” of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this paper, we survey the current landscape of Neural IR research, paying special attention to the use of learned distributed representations of textual units. We highlight the successes of neural IR thus far, catalog obstacles to its wider adoption, and suggest potentially promising directions for future research.

/pdf/neural-information-retrieval-at-the-end-of-the-early-years-144j0hldyz.pdf

Neural information retrieval: at the end of the early years

RTB (Real Time Bidding) is one of the most exciting developments in computational advertising in recent years. It drives transparency and efficiency in the display advertising ecosystem and facilitates the healthy growth of the display advertising industry. It enables advertisers to deliver the right message to the right person at the right time, publishers to better monetize their content by leveraging their website audience, and consumers to view relevant information through personalized ads. However, researchers in computational advertising area have been suffering from lack of publicly available datasets. iPinYou organizes a three-season global RTB algorithm competition in 2013. For each season, there is offline stage and online stage. On the offline stage, iPinYou releases a dataset for model training and reserves a dataset for testing. The dataset includes logs of ad biddings, impressions, clicks, and final conversions. After the whole competition ends, iPinYou organizes and releases all these three datasets for public use. These datasets can support experiments of some important research problems such as bid optimization and CTR estimation. To the best of our knowledge, this is the first publicly available dataset on RTB display advertising. In this paper, we give descriptions of these datasets to further boost the interests of computational advertising research community using this dataset.

/pdf/ipinyou-global-rtb-bidding-algorithm-competition-dataset-3h5trysmd9.pdf

Hairen Liao

Papers

iPinYou Global RTB Bidding Algorithm Competition Dataset