Multi-site data collection and evaluation in spoken language understanding
Lynette Hirschman,M. Bates,Deborah A. Dahl,W. Fisher,J. Garofolo,D. Pallett,K. Hunicke-Smith,Patti Price,Alexander I. Rudnicky,Evelyne Tzoukermann +9 more
- pp 19-24
Reads0
Chats0
TLDR
This work focuses here on selection of training and test data, evaluation of language understanding, and the continuing search for evaluation methods that will correlate well with expected performance of the technology in applications.Abstract:
The Air Travel Information System (ATIS) domain serves as the common task for DARPA spoken language system research and development The approaches and results possible in this rapidly growing area are structured by available corpora, annotations of that data, and evaluation methods Coordination of this crucial infrastructure is the charter of the Multi-Site ATIS Data COllection Working group (MADCOW) We focus here on selection of training and test data, evaluation of language understanding, and the continuing search for evaluation methods that will correlate well with expected performance of the technology in applicationsread more
Citations
More filters
Patent
Intelligent Automated Assistant
Thomas R. Gruber,Adam Cheyer,Dag Kittlaus,Didier Rene Guzzoni,Christopher Dean Brigham,Richard Donald Giuli,Marcello Bastea-Forte,Harry J. Saddler +7 more
TL;DR: In this article, an intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions.
Patent
Using context information to facilitate processing of commands in a virtual assistant
TL;DR: In this article, a virtual assistant uses context information to supplement natural language or gestural input from a user, which helps to clarify the user's intent and reduce the number of candidate interpretations of user's input, and reduces the need for the user to provide excessive clarification input.
Proceedings ArticleDOI
Expanding the scope of the ATIS task: the ATIS-3 corpus
Deborah A. Dahl,Madeleine Bates,Michael Brown,William Fisher,Kate Hunicke-Smith,David Pallett,Christine Pao,Alexander I. Rudnicky,Elizabeth Shriberg +8 more
TL;DR: The migration of the ATIS task to a richer relational database and development corpus (ATIS-3) and the ATis-3 corpus is described, including breakdowns of data by type (e.g. context-independent, context-dependent, and unevaluable) and variations in the data collected at different sites.
Patent
Method and apparatus for building an intelligent automated assistant
Adam Cheyer,Didier Rene Guzzoni +1 more
TL;DR: In this paper, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontologies includes at least one active processing element that models a domain.
Patent
Contextual voice commands
TL;DR: In this paper, techniques and systems for implementing contextual voice commands are described and a physical input that relates the selected data item to an operation in a second context is received, and the operation is performed on the input data item in the second context.
References
More filters
Proceedings ArticleDOI
Evaluation of spoken language systems: the ATIS domain
TL;DR: This paper will address the emerging standards for evaluation of spoken language systems with quantifiable measures essential for comparing results and assessing claims.
Proceedings ArticleDOI
Procedure for quantitatively comparing the syntactic coverage of English grammars
Steven Abney,S. Flickenger,Claudia Gdaniec,C. Grishman,Philip Harrison,Donald Hindle,Robert Ingria,Frederick Jelinek,Judith L. Klavans,Mark Liberman,Mitchell Marcus,Salim Roukos,Beatrice Santorini,Tomek Strzalkowski,Ezra Black +14 more
TL;DR: The problem of quantitatively comparing the performance of different broad-coverage grammars of English has to date resisted solution as discussed by the authors, which is a problem that has been resisted solution.
Proceedings ArticleDOI
Multi-site data collection for a spoken language corpus
TL;DR: A recently collected spoken language corpus for the ATIS (Air Travel Information System) domain is described and the motivation for this effort, the goals, the implementation of a multi-site data collection paradigm, and the accomplishments of MADCOW are summarized.
Proceedings ArticleDOI
Benchmark tests for the DARPA Spoken Language Program
TL;DR: These tests were reported on and discussed in detail at the Spoken Language Systems Technology Workshop held at the Massachusetts Institute of Technology, January 20-22, 1993.