Neural ranking models for information retrieval (IR) use shallow or deep neural networks to rank search results in response to a query. Traditional learning to rank models employ machine learning techniques over hand-crafted IR features. By contrast, neural models learn representations of language from raw text that can bridge the gap between query and document vocabulary. Unlike classical IR models, these new machine learning based approaches are data-hungry, requiring large scale training data before they can be deployed. This tutorial introduces basic concepts and intuitions behind neural IR models, and places them in the context of traditional retrieval models. We begin by introducing fundamental concepts of IR and different neural and non-neural approaches to learning vector representations of text. We then review shallow neural IR methods that employ pre-trained neural term embeddings without learning the IR task end-to-end. We introduce deep neural networks next, discussing popular deep architectures. Finally, we review the current DNN models for information retrieval. We conclude with a discussion on potential future directions for neural IR.

Neural Models for Information Retrieval

A recent “third wave” of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this paper, we survey the current landscape of Neural IR research, paying special attention to the use of learned distributed representations of textual units. We highlight the successes of neural IR thus far, catalog obstacles to its wider adoption, and suggest potentially promising directions for future research.

/pdf/neural-information-retrieval-at-the-end-of-the-early-years-144j0hldyz.pdf

Neural information retrieval: at the end of the early years

The most significant progress in recent years in online display advertising is what is known as the Real-Time Bidding (RTB) mechanism to buy and sell ads. RTB essentially facilitates buying an individual ad impression in real time while it is still being generated from a user's visit. RTB not only scales up the buying process by aggregating a large amount of available inventories across publishers but, most importantly, enables direct targeting of individual users. As such, RTB has fundamentally changed the landscape of digital marketing. Scientifically, the demand for automation, integration and optimisation in RTB also brings new research opportunities in information retrieval, data mining, machine learning and other related fields. In this monograph, an overview is given of the fundamental infrastructure, algorithms, and technical solutions of this new frontier of computational advertising. The covered topics include user response prediction, bid landscape forecasting, bidding algorithms, revenue optimisation, statistical arbitrage, dynamic pricing, and ad fraud detection.

Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting

PubMed is a free search engine for biomedical literature accessed by millions of users from around the world each day With the rapid growth of biomedical literature—about two articles are added every minute on average—finding and retrieving the most relevant papers for a given query is increasingly challenging We present Best Match, a new relevance search algorithm for PubMed that leverages the intelligence of our users and cutting-edge machine-learning technology as an alternative to the traditional date sort order The Best Match algorithm is trained with past user searches with dozens of relevance-ranking signals (factors), the most important being the past usage of an article, publication date, relevance score, and type of article This new algorithm demonstrates state-of-the-art retrieval performance in benchmarking experiments as well as an improved user experience in real-world testing (over 20% increase in user click-through rate) Since its deployment in June 2017, we have observed a significant increase (60%) in PubMed searches with relevance sort order: it now assists millions of PubMed searches each week In this work, we hope to increase the awareness and transparency of this new relevance sort option for PubMed users, enabling them to retrieve information more effectively

/pdf/best-match-new-relevance-search-for-pubmed-bfksdk4za3.pdf

Best Match: New relevance search for PubMed.

Automatic modulation classification facilitates many important signal processing applications. Recently, deep learning models have been adopted in modulation recognition, which outperform traditional machine learning techniques based on hand-crafted features. However, automatic modulation classification is still challenging due to the following reasons. Existing deep learning methods are only applicable to the data of the same distribution. In practical scenarios, data distribution is varying with sampling frequency, thus domains with different sampling rates are formed. Besides, it is difficult to construct large-scale well-annotated datasets for all domains of interest. We define the domain with sufficient data as the source domain, while the domain with insufficient data as the target domain. Obviously, the classification model performs weakly in the target domain. To address these challenges, we propose an adversarial transfer learning architecture (ATLA), incorporating adversarial training and knowledge transfer in a unified way. Adversarial training performs an asymmetric mapping between domains and reduces the domain shift. Knowledge transfer is used to mine prior knowledge from the source domain. Experimental results demonstrate that the proposed ATLA substantially boosts the performance of the target model, which outperforms the existing parameter-transfer approach. With half of the training data reduced, the target model achieves competitive recognition accuracy to supervised learning. With one-tenth of training data, the promoted accuracy is up to 17.3% points.

Adversarial Transfer Learning for Deep Learning Based Automatic Modulation Classification

Recent years have observed a significant progress in information retrieval and natural language processing with deep learning technologies being successfully applied into almost all of their major tasks. The key to the success of deep learning is its capability of accurately learning distributed representations (vector representations or structured arrangement of them) of natural language expressions such as sentences, and effectively utilizing the representations in the tasks. This tutorial aims at summarizing and introducing the results of recent research on deep learning for information retrieval, in order to stimulate and foster more significant research and development work on the topic in the future. The tutorial mainly consists of three parts. In the first part, we introduce the fundamental techniques of deep learning for natural language processing and information retrieval, such as word embedding, recurrent neural networks, and convolutional neural networks. In the second part, we explain how deep learning, particularly representation learning techniques, can be utilized in fundamental NLP and IR problems, including matching, translation, classification, and structured prediction. In the third part, we describe how deep learning can be used in specific application tasks in details. The tasks are search, question answering (from either documents, database, or knowledge base), and image retrieval.

Zhengdong Lu

Papers

Deep Learning for Information Retrieval