Neural Belief Tracker: Data-Driven Dialogue State Tracking
Nikola Mrkšić,Diarmuid Ó Séaghdha,Tsung-Hsien Wen,Blaise Thomson,Steve Young +4 more
- Vol. 1, pp 1777-1788
TLDR
This work proposes a novel Neural Belief Tracking (NBT) framework which overcomes past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.Abstract:
One of the core components of modern spoken dialogue systems is the belief tracker, which estimates the user’s goal at every step of the dialogue. However, most current approaches have difficulty scaling to larger, more complex dialogue domains. This is due to their dependency on either: a) Spoken Language Understanding models that require large amounts of annotated training data; or b) hand-crafted lexicons for capturing some of the linguistic variation in users’ language. We propose a novel Neural Belief Tracking (NBT) framework which overcomes these problems by building on recent advances in representation learning. NBT models reason over pre-trained word vectors, learning to compose them into distributed representations of user utterances and dialogue context. Our evaluation on two datasets shows that this approach surpasses past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.read more
Citations
More filters
Posted Content
CREDIT: Coarse-to-Fine Sequence Generation for Dialogue State Tracking.
TL;DR: This paper employs a structured state representation and cast dialogue state tracking as a sequence generation problem and proposes a generative state tracking method, CREDIT, which does not rely on pre-defined dialogue ontology enumerating all possible slot values.
Posted Content
Scaling Multi-Domain Dialogue State Tracking via Query Reformulation
TL;DR: A novel approach to dialogue state tracking and referring expression resolution tasks, which model the reference resolution as a dialogue context-aware user query reformulation task – the dialog state is serialized to a sequence of natural language tokens representing the conversation.
Posted Content
Improving Limited Labeled Dialogue State Tracking with Self-Supervision
TL;DR: This paper presents and investigates two self-supervised objectives: preserving latent consistency and modeling conversational behavior, and encourages a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario.
Posted Content
Reinforcement Learning-based Dialogue Guided Event Extraction to Exploit Argument Relations.
TL;DR: This paper used reinforcement learning and incremental learning to extract multiple arguments via a multi-turned, iterative process, which leverages knowledge of the already extracted arguments of the same sentence to determine the role of arguments that would be difficult to decide individually.
A Multi-Task Approach to Incremental Dialogue State Tracking
TL;DR: This paper presents the design of the incremental dialogue state tracker in detail and provides evaluation against the well known Dialogue State Tracking Challenge 2 (DSTC2) dataset and finds that the Multi-Task Learning-based model achieves state-of-the-art results for incremental processing.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Proceedings ArticleDOI
Glove: Global Vectors for Word Representation
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.