Proceedings ArticleDOI
Slot Filling with Weighted Multi-Encoders for Out-of-Domain Values.
Kobayashi Yuka,Yoshida Takami,Kenji Iwata,Hiroshi Fujimura +3 more
- pp 854-858
TLDR
A new method for slot filling of out-ofdomain (OOD) slot values, which are not included in the training data, in spoken dialogue systems, using two encoders, which distinctly encode contexts and keywords, respectively.Abstract:
This paper proposes a new method for slot filling of out-ofdomain (OOD) slot values, which are not included in the training data, in spoken dialogue systems. Word embeddings have been proposed to estimate the OOD slot values included in the word embedding model from keyword information. At the same time, context information is an important clue for estimation because the values in a given slot tend to appear in similar contexts. The proper use of either or both keyword and context information depends on the sentence. Conventional methods input a whole sentence into an encoder and extract important clues by the attention mechanism. However, it is difficult to properly distinguish context and keyword information from the encoder outputs because these two features are already mixed. Our proposed method uses two encoders, which distinctly encode contexts and keywords, respectively. The model calculates weights for the two encoders based on a user utterance and estimates a slot with weighted outputs from the two encoders. Experimental results show that the proposed method achieves a 50% relative improvement in F1 score compared with a baseline model, which detects slot values from user utterances and estimates slots at once with a single encoder.read more
Citations
More filters
Journal ArticleDOI
ARoBERT: An ASR Robust Pre-Trained Language Model for Spoken Language Understanding
TL;DR: This work presents ARoBERT, an ASR Robust BERT model, which can be fine-tuned to solve a variety of SLU tasks with noisy inputs, and proposes two novel self-supervised pre-training tasks for ARoberT, namely Phonetically-aware Masked Language Modeling and ASR Model-adaptive Masked language Modeling.
Posted Content
Interactive teaching for conversational AI
Qing Ping,Feiyang Niu,Govind Thattai,Joel Chengottusseriyil,Qiaozi Gao,Aishwarya N. Reganti,Prashanth Rajagopal,Gokhan Tur,Dilek Hakkani-Tur,Prem Natarajan +9 more
TL;DR: A new Teachable AI system that is capable of learning new language nuggets called concepts, directly from end users using live interactive teaching sessions, and it is demonstrated that this method is very promising in leading way to build more adaptive and personalized language understanding models.
Journal ArticleDOI
A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario
TL;DR: The results suggest that joint NLU models trained with short slots yield better results than those trained with long slots for the slot filling task and indicate that short slots could be a better choice for the dialog system because of their simplicity.
References
More filters
Proceedings ArticleDOI
Glove: Global Vectors for Word Representation
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Posted Content
Efficient Estimation of Word Representations in Vector Space
TL;DR: This paper proposed two novel model architectures for computing continuous vector representations of words from very large data sets, and the quality of these representations is measured in a word similarity task and the results are compared to the previously best performing techniques based on different types of neural networks.
Proceedings Article
Neural Machine Translation by Jointly Learning to Align and Translate
TL;DR: It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Proceedings ArticleDOI
Learning Phrase Representations using RNN Encoder--Decoder for Statistical Machine Translation
Kyunghyun Cho,Bart van Merriënboer,Caglar Gulcehre,Dzmitry Bahdanau,Fethi Bougares,Holger Schwenk,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +8 more
TL;DR: In this paper, the encoder and decoder of the RNN Encoder-Decoder model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.
Posted Content
Neural Machine Translation by Jointly Learning to Align and Translate
TL;DR: In this paper, the authors propose to use a soft-searching model to find the parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Related Papers (5)
Adaptive quantization with balanced pixel-domain distortion distribution in image processing
Xiang Yu,Dake He,En-hui Yang +2 more