Neural Belief Tracker: Data-Driven Dialogue State Tracking
Nikola Mrkšić,Diarmuid Ó Séaghdha,Tsung-Hsien Wen,Blaise Thomson,Steve Young +4 more
- Vol. 1, pp 1777-1788
TLDR
This work proposes a novel Neural Belief Tracking (NBT) framework which overcomes past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.Abstract:
One of the core components of modern spoken dialogue systems is the belief tracker, which estimates the user’s goal at every step of the dialogue. However, most current approaches have difficulty scaling to larger, more complex dialogue domains. This is due to their dependency on either: a) Spoken Language Understanding models that require large amounts of annotated training data; or b) hand-crafted lexicons for capturing some of the linguistic variation in users’ language. We propose a novel Neural Belief Tracking (NBT) framework which overcomes these problems by building on recent advances in representation learning. NBT models reason over pre-trained word vectors, learning to compose them into distributed representations of user utterances and dialogue context. Our evaluation on two datasets shows that this approach surpasses past limitations, matching the performance of state-of-the-art models which rely on hand-crafted semantic lexicons and outperforming them when such lexicons are not provided.read more
Citations
More filters
Proceedings ArticleDOI
Action State Update Approach to Dialogue Management
TL;DR: In this article, an action state update approach is proposed for utterance interpretation, featuring a statistically trained binary classifier used to detect dialogue state update actions in the text of a user utterance.
Journal ArticleDOI
Information Extraction and Human-Robot Dialogue towards Real-life Tasks A Baseline Study with the MobileCS Dataset
TL;DR: A baseline study of the two tasks with the MobileCS dataset, which consists of real-world dialog transcripts between real users and customer-service staffs from China Mobile, and how the two baselines are constructed, the problems en-countered, and the results are introduced.
Proceedings ArticleDOI
Deep learning for language understanding of mental health concepts derived from Cognitive Behavioural Therapy.
Lina Maria Rojas-Barahona,Bo-Hsiang Tseng,Yinpei Dai,Clare Mansfield,Osman Ramadan,Stefan Ultes,Michael Crawford,Milica Gasic +7 more
TL;DR: In this paper, the authors define a mental health ontology based on the CBT principles, annotate a large corpus where this phenomena is exhibited and perform understanding using deep learning and distributed representations.
Book
Data-Driven Language Understanding for Spoken Dialogue Systems
TL;DR: The proposed Neural Belief Tracking model forsakes the use of standard one-hot n-gram representations used in Natural Language Processing in favour of distributed representations of user utterances, dialogue context and domain ontologies, and the proposed ATTRACT-REPEL algorithm boosts the semantic content of existing word vectors while simultaneously inducing high-quality cross-lingual word vector spaces.
Posted Content
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems
Evgeniia Razumovskaia,Goran Glavaš,Olga Majewska,Edoardo Maria Ponti,Anna Korhonen,Ivan Vulić +5 more
TL;DR: The authors provide an extensive overview of existing methods and resources in multilingual task-oriented dialogue (ToD) as an entry point to this exciting and emerging field and find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Proceedings ArticleDOI
Glove: Global Vectors for Word Representation
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.