Neural Belief Tracker: Data-Driven Dialogue State Tracking
Nikola Mrkšić,Diarmuid Ó Séaghdha,Tsung-Hsien Wen,Blaise Thomson,Steve Young +4 more
- Vol. 1, pp 1777-1788
TLDR
This work proposes a novel Neural Belief Tracking (NBT) framework which overcomes past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.Abstract:
One of the core components of modern spoken dialogue systems is the belief tracker, which estimates the user’s goal at every step of the dialogue. However, most current approaches have difficulty scaling to larger, more complex dialogue domains. This is due to their dependency on either: a) Spoken Language Understanding models that require large amounts of annotated training data; or b) hand-crafted lexicons for capturing some of the linguistic variation in users’ language. We propose a novel Neural Belief Tracking (NBT) framework which overcomes these problems by building on recent advances in representation learning. NBT models reason over pre-trained word vectors, learning to compose them into distributed representations of user utterances and dialogue context. Our evaluation on two datasets shows that this approach surpasses past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.read more
Citations
More filters
Proceedings ArticleDOI
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue
TL;DR: This article propose a self-training framework for dialogue pre-training, which distills future knowledge to the representation of the previous dialogue context using a selftraining framework. But the model is limited to task-oriented dialogues.
Posted Content
SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain
Ruochen Zhang,Carsten Eickhoff +1 more
TL;DR: In this article, the authors proposed a task formulation where, given paragraphs of commentary of a game at different timestamps, the system is asked to recognize the occurrence of in-game events.
Posted Content
State-Machine-Based Dialogue Agents with Few-Shot Contextual Semantic Parsers.
TL;DR: A methodology and toolkit for creating a rule-based multi-domain conversational agent for transactions from language annotations of the domains' database schemas and APIs and a couple of hundreds of annotated human dialogues, which achieves over 71% turn-by-turn slot accuracy on the MultiWOZ benchmark.
Proceedings Article
Hi-dst: a hierarchical approach for scalable and extensible dialogue state tracking
Proceedings Article
Addressing Domain Changes in Task-oriented Conversational Agents through Dialogue Adaptation
Tiziano Labruna,Bernardo Magnini +1 more
TL;DR: The authors proposed a dialogue adaptation approach based on fine-tuning a generative language model on domain changes, showing that a significant reduction of performance decrease can be obtained by automatic adaptation of training dialogues.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Proceedings ArticleDOI
Glove: Global Vectors for Word Representation
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.