Deep Interest Network for Click-Through Rate Prediction
Guorui Zhou,Xiaoqiang Zhu,Chenru Song,Ying Fan,Han Zhu,Xiao Ma,Yan Yanghui,Junqi Jin,Han Li,Kun Gai +9 more
- pp 1059-1068
Reads0
Chats0
TLDR
A novel model: Deep Interest Network (DIN) is proposed which tackles this challenge by designing a local activation unit to adaptively learn the representation of user interests from historical behaviors with respect to a certain ad.Abstract:
Click-through rate prediction is an essential task in industrial applications, such as online advertising. Recently deep learning based models have been proposed, which follow a similar Embedding&MLP paradigm. In these methods large scale sparse input features are first mapped into low dimensional embedding vectors, and then transformed into fixed-length vectors in a group-wise manner, finally concatenated together to fed into a multilayer perceptron (MLP) to learn the nonlinear relations among features. In this way, user features are compressed into a fixed-length representation vector, in regardless of what candidate ads are. The use of fixed-length vector will be a bottleneck, which brings difficulty for Embedding&MLP methods to capture user's diverse interests effectively from rich historical behaviors. In this paper, we propose a novel model: Deep Interest Network (DIN) which tackles this challenge by designing a local activation unit to adaptively learn the representation of user interests from historical behaviors with respect to a certain ad. This representation vector varies over different ads, improving the expressive ability of model greatly. Besides, we develop two techniques: mini-batch aware regularization and data adaptive activation function which can help training industrial deep networks with hundreds of millions of parameters. Experiments on two public datasets as well as an Alibaba real production dataset with over 2 billion samples demonstrate the effectiveness of proposed approaches, which achieve superior performance compared with state-of-the-art methods. DIN now has been successfully deployed in the online display advertising system in Alibaba, serving the main traffic.read more
Citations
More filters
Proceedings ArticleDOI
Distributed Equivalent Substitution Training for Large-Scale Recommender Systems
Haidong Rong,Yangzihao Wang,Feihu Zhou,Junjie Zhai,Haiyang Wu,Rui Lan,Fan Li,Han Zhang,Yuekui Yang,Zhenyu Guo,Di Wang +10 more
TL;DR: Distributed Equivalent Substitution (DES) as discussed by the authors replaces weights-rich operators with the computationally equivalent sub-operators and aggregates partial results instead of transmitting the huge sparse weights directly through the network.
Proceedings ArticleDOI
Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning
Chengfei Lv,Chaoyue Niu,Renjie Gu,Xiaotang Jiang,Zhaode Wang,Bin Liu,Ziqi Wu,Qiulin Yao,Congyu Huang,Panos Huang,Tao Huang,Hui Shu,Jinde Song,Bingzhang Zou,Peng Lan,Guohuan Xu,Fei Wu,Shaojie Tang,Fan Wu,Guihai Chen +19 more
TL;DR: This work adopts device-cloud collaborative ML and builds the first end-to-end and general-purpose system, called Walle, which consists of a deployment platform, distributing ML tasks to billion-scale devices in time; a data pipeline, preparing task input; and a compute container, providing a cross-platform and high-performance execution environment, while facilitating daily task iteration.
Journal ArticleDOI
AIRec: Attentive intersection model for tag-aware recommendation
TL;DR: Award-aware recommender systems (TRS) utilize rich tagging information to better depict user portraits and item features, but existing methods construct user representations through either explicit tagging behaviors or implicit interacted items, which is inadequate to capture multi-aspect user preferences.
Posted Content
GRN: Generative Rerank Network for Context-wise Recommendation
TL;DR: This work designs the evaluator, which applies Bi-LSTM and self-attention mechanism to model the contextual information in the labeled final ranking list and predicts the interaction probability of each item more precisely, and proposes a new context-wise reranking framework named GRN.
Journal ArticleDOI
Efficient Federated Learning With Spike Neural Networks for Traffic Sign Recognition
TL;DR: This work introduces powerful Spike Neural Networks (SNNs) into traffic sign recognition for energy-efficient and fast model training, which is the next generation of neural networks and is practical and well-fitted to IoV scenarios.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal ArticleDOI
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings ArticleDOI
Densely Connected Convolutional Networks
TL;DR: DenseNet as mentioned in this paper proposes to connect each layer to every other layer in a feed-forward fashion, which can alleviate the vanishing gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.