Survey of Available Datasets for Designing Task Oriented Dialogue Agents
20 Dec 2019-
TL;DR: In this paper, the authors present a survey of publicly available datasets and their applicability for designing modern task-oriented dialogue agents, including task oriented dialogue agents that span over multiple domains to accomplish a complex user goal.
Abstract: Dialogue Systems are increasingly popular with the recent advances in neural approaches and NLP applied to conversational AI. Alexa, Siri, Cortana, Google Mini are handily used by many users to do small tasks and control their home appliances in hands free style. Enterprises are also deploying 24 × 7 dialogue agent in place of traditional customer support to increase user engagement and improve their processes. Dialogue Systems are also augmented with Robots to improve human-robot dialogues.Conversational Agents are classified into two main types: Social bots/Chitchat bots and Task Oriented Dialogue Agents. Social bots aim to engage user with unstructured human conversations. These dialogue agents don’t have fixed aim to complete and focus more on carrying out open domain conversations. For example ALIZA, Microsoft XiaoIce etcOn the other hand, Task oriented dialogue agents help user to accomplish certain tasks in specific domains like Restaurant booking, Flight reservation, customer support etc. These are popularly used in controlling home appliances and carrying out simple tasks by users in day to day life. Siri, Alexa, Google Mini, Cortana are task oriented dialogue agents. There is increasing interest in building task completion dialogue agents that span over multiple sub-domains to accomplish a complex user goal.With the increasing acceptance of Dialogue Agents, there is need of high-quality, large-scale dialogue datasets for better performance of task oriented dialogue agent in changing environment. Neural approaches are applied to design intelligent dialogue agents frequently which require very large datasets. However, there are following challenges while building intelligent task completion dialogue systems. Firstly, there are a lot of datasets available for chit-chat bots but they are not directly relevant to task oriented systems. Secondly, to scale out the system to new domains with limited in-domain data.In this paper, we studied different data collection methods, important characteristics of dialogue datasets and their potential uses. This paper presents a survey of publicly available datasets and their applicability for designing modern task - oriented dialogue agents.
Citations
More filters
01 Jan 2023
TL;DR: In this paper , the authors presented the analysis of the incidence of virtual assistance with the use of the chat bot tool to contribute to customer loyalty in companies providing fixed Internet service, and concluded that the virtual assistant strategy reached an acceptance rate of 71% and generated the beginning of customer loyalty for after-sales services in fixed Internet companies.
Abstract: AbstractThe ability of companies to provide quality products and services motivates organizations to develop business strategies with investment in technology. This research presents the analysis of the incidence of virtual assistance with the use of the chat bot tool to contribute to customer loyalty in companies providing fixed Internet service. The objective is framed in establishing post-sales strategies in the fixed Internet service, by means of customer retention and satisfaction tools. The type of research is descriptive, and the work contributes with the situational explanation of the problem and possible ways to solve them. The structured survey technique was used to 406 users of the Internet service in the home modality in the provinces of Ecuador. The study variables determined are customer loyalty, after-sales strategies, and virtual assistance, according to the national context. The results show that 45% of users remain with the same provider for more than 12 months. It is concluded that the virtual assistant strategy reaches an acceptance rate of 71% and generates the beginning of customer loyalty for after-sales services in fixed Internet companies.KeywordsVirtual assistanceAfter-sales strategiesCustomer loyaltyFixed internet
References
More filters
24 Jun 1990
TL;DR: This pilot marks the first full-scale attempt to collect a corpus to measure progress in Spoken Language Systems that include both a speech and natural language component and provides guidelines for future efforts.
Abstract: Speech research has made tremendous progress in the past using the following paradigm:• define the research problem,• collect a corpus to objectively measure progress, and• solve the research problem.Natural language research, on the other hand, has typically progressed without the benefit of any corpus of data with which to test research hypotheses. We describe the Air Travel Information System (ATIS) pilot corpus, a corpus designed to measure progress in Spoken Language Systems that include both a speech and natural language component. This pilot marks the first full-scale attempt to collect such a corpus and provides guidelines for future efforts.
842 citations
01 Jun 2014
TL;DR: The results suggest that while large improvements on a competitive baseline are possible, trackers are still prone to degradation in mismatched conditions and ensemble learning demonstrates the most accurate tracking can be achieved by combining multiple trackers.
Abstract: A spoken dialog system, while communicating with a user, must keep track of what the user wants from the system at each step. This process, termed dialog state tracking, is essential for a successful dialog system as it directly informs the system’s actions. The first Dialog State Tracking Challenge allowed for evaluation of different dialog state tracking techniques, providing common testbeds and evaluation suites. This paper presents a second challenge, which continues this tradition and introduces some additional features ‐ a new domain, changing user goals and a richer dialog state. The challenge received 31 entries from 9 research groups. The results suggest that while large improvements on a competitive baseline are possible, trackers are still prone to degradation in mismatched conditions. An investigation into ensemble learning demonstrates the most accurate tracking can be achieved by combining multiple trackers.
655 citations
Posted Content•
TL;DR: The Multi-Domain Wizard-of-Oz dataset (MultiWOZ) as discussed by the authors is a fully-labeled collection of human-human written conversations spanning over multiple domains and topics.
Abstract: Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available. To address this fundamental obstacle, we introduce the Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics. At a size of $10$k dialogues, it is at least one order of magnitude larger than all previous annotated task-oriented corpora. The contribution of this work apart from the open-sourced dataset labelled with dialogue belief states and dialogue actions is two-fold: firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The proposed data-collection pipeline is entirely based on crowd-sourcing without the need of hiring professional annotators; secondly, a set of benchmark results of belief tracking, dialogue act and response generation is reported, which shows the usability of the data and sets a baseline for future studies.
623 citations
Posted Content•
TL;DR: The machine learning architecture of the Snips Voice Platform is presented, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices that is fast and accurate while enforcing privacy by design, as no personal user data is ever collected.
Abstract: This paper presents the machine learning architecture of the Snips Voice Platform, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices. The embedded inference is fast and accurate while enforcing privacy by design, as no personal user data is ever collected. Focusing on Automatic Speech Recognition and Natural Language Understanding, we detail our approach to training high-performance Machine Learning models that are small enough to run in real-time on small devices. Additionally, we describe a data generation procedure that provides sufficient, high-quality training data without compromising user privacy.
566 citations
01 Jul 2019
TL;DR: The Multimodal EmotionLines Dataset (MELD) as discussed by the authors is a large-scale multimodal multi-party emotional conversational database containing more than two speakers per dialogue.
Abstract: Emotion recognition in conversations is a challenging task that has recently gained popularity due to its potential applications. Until now, however, a large-scale multimodal multi-party emotional conversational database containing more than two speakers per dialogue was missing. Thus, we propose the Multimodal EmotionLines Dataset (MELD), an extension and enhancement of EmotionLines. MELD contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends. Each utterance is annotated with emotion and sentiment labels, and encompasses audio, visual and textual modalities. We propose several strong multimodal baselines and show the importance of contextual and multimodal information for emotion recognition in conversations. The full dataset is available for use at http://affective-meld.github.io.
498 citations