Proceedings ArticleDOI
Semantic Matching Research based on Interactive Multi-head Attention Mechanism
Yulin Liao,Tongyan Li +1 more
Reads0
Chats0
TLDR
In this article , a multi-head attention mechanism is added to the Siamease model based on bidirectional LSTM to solve the problem of insufficient semantic extraction, and on this basis, it is proposed to add fine-grained matching information in the form of an interactive multihop attention mechanism.Abstract:
Semantic matching plays a crucial technical supporting role in question answering systems. For semantic matching, it is mainly based on neural networks to solve the sentence representation and interaction in semantic matching. The Siamease network structure is a commonly used structure in semantic matching. In view of the problem of information exchange and semantic extraction not saving points in the independent training of the network structure using shared parameters for the input two sequences. Therefore, in this paper, a multi-head attention mechanism is added to the Siamease model based on bidirectional LSTM to solve the problem of insufficient semantic extraction, and on this basis, it is proposed to add fine-grained matching information in the form of an interactive multi-head attention mechanism to solve the interaction problem. Experimental results show that the performance of the model is further improved compared to previous deep learning models.read more
References
More filters
Proceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
TL;DR: This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Journal ArticleDOI
A vector space model for automatic indexing
Gerard Salton,A. Wong,C. S. Yang +2 more
TL;DR: An approach based on space density computations is used to choose an optimum indexing vocabulary for a collection of documents, demonstating the usefulness of the model.
Proceedings ArticleDOI
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
TL;DR: The authors introduced the Tree-LSTM, a generalization of LSTMs to tree-structured network topologies, which outperformed all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task 1) and sentiment classification (Stanford Sentiment Treebank).
Proceedings ArticleDOI
Learning deep structured semantic models for web search using clickthrough data
TL;DR: A series of new latent semantic models with a deep structure that project queries and documents into a common low-dimensional space where the relevance of a document given a query is readily computed as the distance between them are developed.