Proceedings ArticleDOI
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Fei Sun,Jun Liu,Jian Wu,Changhua Pei,Xiao Lin,Wenwu Ou,Peng Jiang +6 more
- pp 1441-1450
Reads0
Chats0
TLDR
BERT4Rec as discussed by the authors employs the deep bidirectional self-attention to model user behavior sequences, predicting the random masked items in the sequence by jointly conditioning on their left and right context.Abstract:
Modeling users' dynamic preferences from their historical behaviors is challenging and crucial for recommendation systems. Previous methods employ sequential neural networks to encode users' historical interactions from left to right into hidden representations for making recommendations. Despite their effectiveness, we argue that such left-to-right unidirectional models are sub-optimal due to the limitations including: \begin enumerate* [label=series\itshape\alph*\upshape)] \item unidirectional architectures restrict the power of hidden representation in users' behavior sequences; \item they often assume a rigidly ordered sequence which is not always practical. \end enumerate* To address these limitations, we proposed a sequential recommendation model called BERT4Rec, which employs the deep bidirectional self-attention to model user behavior sequences. To avoid the information leakage and efficiently train the bidirectional model, we adopt the Cloze objective to sequential recommendation, predicting the random masked items in the sequence by jointly conditioning on their left and right context. In this way, we learn a bidirectional representation model to make recommendations by allowing each item in user historical behaviors to fuse information from both left and right sides. Extensive experiments on four benchmark datasets show that our model outperforms various state-of-the-art sequential models consistently.read more
Citations
More filters
Posted Content
Graph Neural Networks in Recommender Systems: A Survey
TL;DR: This article provides a taxonomy of GNN-based recommendation models according to the types of information used and recommendation tasks and systematically analyze the challenges of applying GNN on different types of data.
Proceedings ArticleDOI
Behavior sequence transformer for e-commerce recommendation in Alibaba
TL;DR: This paper proposes to use the powerful Transformer model to capture the sequential signals underlying users' behavior sequences for recommendation in Alibaba and demonstrates the superiority of the proposed model.
Proceedings ArticleDOI
S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
Kun Zhou,Hui Wang,Wayne Xin Zhao,Yutao Zhu,Sirui Wang,Fuzheng Zhang,Zhongyuan Wang,Ji-Rong Wen +7 more
TL;DR: Li et al. as mentioned in this paper proposed a self-supervised learning for sequential recommendation based on the self-attentive neural architecture, which utilizes the intrinsic data correlation to derive self-vision signals and enhance the data representations via pre-training methods for improving sequential recommendation.
Proceedings ArticleDOI
Next-item Recommendation with Sequential Hypergraphs
TL;DR: The proposed model can significantly outperform the state-of-the-art in predicting the next interesting item for each user and is equipped with a fusion layer to incorporate both the dynamic item embedding and short-term user intent to the representation of each interaction.
Proceedings ArticleDOI
Contrastive Learning for Sequential Recommendation
TL;DR: A novel multi-task framework called Contrastive Learning for Sequential Recommendation (CL4SRec) is proposed, which not only takes advantage of the traditional next item prediction task but also utilizes the contrastive learning framework to derive self-supervision signals from the original user behavior sequences.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Proceedings ArticleDOI
Learning Phrase Representations using RNN Encoder--Decoder for Statistical Machine Translation
Kyunghyun Cho,Bart van Merriënboer,Caglar Gulcehre,Dzmitry Bahdanau,Fethi Bougares,Holger Schwenk,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +8 more
TL;DR: In this paper, the encoder and decoder of the RNN Encoder-Decoder model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.
Journal ArticleDOI
Matrix Factorization Techniques for Recommender Systems
TL;DR: As the Netflix Prize competition has demonstrated, matrix factorization models are superior to classic nearest neighbor techniques for producing product recommendations, allowing the incorporation of additional information such as implicit feedback, temporal effects, and confidence levels.
Proceedings ArticleDOI
Item-based collaborative filtering recommendation algorithms
TL;DR: This paper analyzes item-based collaborative ltering techniques and suggests that item- based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality than the best available userbased algorithms.