On Sampled Metrics for Item Recommendation
Walid Krichene,Steffen Rendle +1 more
- pp 1748-1757
Reads0
Chats0
TLDR
It is shown that sampled metrics are inconsistent with their exact version, in the sense that they do not persist relative statements, and it is suggested that sampling should be avoided for metric calculation, however if an experimental study needs to sample, the proposed corrections can improve the quality of the estimate.Abstract:
The task of item recommendation requires ranking a large catalogue of items given a context. Item recommendation algorithms are evaluated using ranking metrics that depend on the positions of relevant items. To speed up the computation of metrics, recent work often uses sampled metrics where only a smaller set of random items and the relevant items are ranked. This paper investigates sampled metrics in more detail and shows that they are inconsistent with their exact version, in the sense that they do not persist relative statements, e.g., recommender A is better than B, not even in expectation. Moreover, the smaller the sampling size, the less difference there is between metrics, and for very small sampling size, all metrics collapse to the AUC metric. We show that it is possible to improve the quality of the sampled metrics by applying a correction, obtained by minimizing different criteria such as bias or mean squared error. We conclude with an empirical evaluation of the naive sampled metrics and their corrected variants. To summarize, our work suggests that sampling should be avoided for metric calculation, however if an experimental study needs to sample, the proposed corrections can improve the quality of the estimate.read more
Citations
More filters
Proceedings ArticleDOI
Learning Intents behind Interactions with Knowledge Graph for Recommendation
Xiang Wang,Tinglin Huang,Dingxian Wang,Yancheng Yuan,Zhenguang Liu,Xiangnan He,Tat-Seng Chua +6 more
TL;DR: Wang et al. as mentioned in this paper explored intents behind a user-item interaction by using auxiliary item knowledge, and proposed a new model, Knowledge Graph-based Intent Network (KGIN), which model each intent as an attentive combination of KG relations, encouraging the independence of different intents for better model capability and interpretability.
Proceedings ArticleDOI
Learning Intents behind Interactions with Knowledge Graph for Recommendation
Xiang Wang,Tinglin Huang,Dingxian Wang,Yancheng Yuan,Zhenguang Liu,Xiangnan He,Tat-Seng Chua +6 more
TL;DR: Huang et al. as discussed by the authors proposed a knowledge graph-based intent network (KGIN) to model each intent as an attentive combination of KG relations, encouraging the independence of different intents.
Proceedings ArticleDOI
Contrastive Learning for Sequential Recommendation
TL;DR: A novel multi-task framework called Contrastive Learning for Sequential Recommendation (CL4SRec) is proposed, which not only takes advantage of the traditional next item prediction task but also utilizes the contrastive learning framework to derive self-supervision signals from the original user behavior sequences.
Posted Content
RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms
Wayne Xin Zhao,Shanlei Mu,Yupeng Hou,Zihan Lin,Kaiyuan Li,Yushuo Chen,Yujie Lu,Hui Wang,Changxin Tian,Xingyu Pan,Yingqian Min,Zhichao Feng,Xinyan Fan,Xu Chen,Pengfei Wang,Wendi Ji,Yaliang Li,Xiaoling Wang,Ji-Rong Wen +18 more
TL;DR: A unified, comprehensive and efficient recommender system library called RecBole (pronounced as [rEk'boUl@r]), which provides a unified framework to develop and reproduce recommendation algorithms for research purpose and provides a series of auxiliary functions, tools, and scripts to facilitate the use of this library.
Proceedings ArticleDOI
Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer
TL;DR: Wang et al. as discussed by the authors proposed a new framework Temporal Graph Sequential Recommender (TGSRec) upon a defined continuous-time bipartite graph, which can simultaneously capture collaborative signals from both users and items, as well as considering temporal dynamics inside sequential patterns.
References
More filters
Posted Content
Efficient Estimation of Word Representations in Vector Space
TL;DR: This paper proposed two novel model architectures for computing continuous vector representations of words from very large data sets, and the quality of these representations is measured in a word similarity task and the results are compared to the previously best performing techniques based on different types of neural networks.
Proceedings ArticleDOI
Item-based collaborative filtering recommendation algorithms
TL;DR: This paper analyzes item-based collaborative ltering techniques and suggests that item- based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality than the best available userbased algorithms.
Proceedings ArticleDOI
Neural Collaborative Filtering
TL;DR: This work strives to develop techniques based on neural networks to tackle the key problem in recommendation --- collaborative filtering --- on the basis of implicit feedback, and presents a general framework named NCF, short for Neural network-based Collaborative Filtering.
Journal ArticleDOI
The MovieLens Datasets: History and Context
TL;DR: The history of MovieLens and the MovieLens datasets is documents, including a discussion of lessons learned from running a long-standing, live research platform from the perspective of a research organization, and best practices and limitations of using the Movie Lens datasets in new research are documented.
Proceedings ArticleDOI
Collaborative Filtering for Implicit Feedback Datasets
TL;DR: This work identifies unique properties of implicit feedback datasets and proposes treating the data as indication of positive and negative preference associated with vastly varying confidence levels, which leads to a factor model which is especially tailored for implicit feedback recommenders.