An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Systems and methods are provided for receiving speech and non-speech communications of natural language questions and/or commands, transcribing the speech and non-speech communications to textual messages, and executing the questions and/or commands. The invention applies context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for one or more users presenting questions or commands across multiple domains. The systems and methods creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context of the speech and non-speech communications and presenting the expected results for a particular question or command.

Systems and methods for responding to natural language speech utterance

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Mobile systems and methods of supporting natural language human-machine interactions

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

An enhanced natural language information retrieval technique tokenizes an incoming query, comparing the tokenized representation against a collection of query templates. Query templates include a first portion having one or more query patterns representative of a query and in a form suitable for matching the tokenized representation of an incoming query. Query templates also include one or more information retrieval commands that are designed to return information relevant to those query patterns in its first portion. The enhanced natural language information retrieval technique selects those query templates that are determined to be most relevant to the incoming query (via its tokenized representation) and initiates one or more information retrieval commands associated with the selected query templates.

Natural language query processing

This patent describes a novel system, method, and program product that are used in interactive natural language dialog. One or more presentation managers operating on a computer system present information from the computer system to one or more users over network interface(s) and accept queries from the users using one or more known input/output modalities (e.g. Speech, typed in text, pointing devices, etc.). A natural language parser parses one or more natural language phrases received over one or more of the network interfaces by one or more of the presentation managers into one or more logical forms (parsed user input), each logical form having a grammatical and structural organization. A dialog manager module maintains and directs interactive sessions between each of the users and the computer system. The dialog manager receives logical forms from one or more of the presentation managers and sends these to a taxonomical mapping process which matches the items of interest to the user against the content organization in the content database to match business categories and sends modified logical forms back to the dialog manager.

System, method and program product for interactive natural dialog

This paper describes the evaluation of a natural language dialog based navigation system (HappyAssistant) that helps users access e-commerce sites to find relevant information about products and services. The prototype system leverages technologies in natural language processing and human computer interaction to create a faster and more intuitive way of interacting with websites, especially for the less experienced users. The result of a comparative study shows that users prefer the natural language enabled navigation two to one over the menu driven navigation. In addition, the study confirmed the efficiency of using natural language dialog in terms of the number of clicks and the amount of time required to obtain the relevant information. In the case study, comparing to the menu driven system, the average number of clicks used in the natural language system was reduced by 63.2% and the average time was reduced by 33.3%.

/pdf/comparative-evaluation-of-a-natural-language-dialog-based-28thjxhhu5.pdf

Comparative evaluation of a natural language dialog based system and a menu driven system for information access: a case study

This paper describes the evaluation of a natural language dialog-based navigation system (HappyAssistant) that helps users access e-commerce sites to find relevant information about products and services. The prototype system leverages technologies in natural language processing and human-computer interaction to create a faster and more intuitive way of interacting with websites, especially for less experienced users. The result of a comparative study shows that users prefer the natural language-enabled navigation two to one over the menu driven navigation. In addition, the study confirmed the efficiency of using natural language dialog in terms of the number of clicks and the amount of time required to obtain the relevant information. In the case study, as compared to the menu driven system, the average number of clicks used in the natural language system was reduced by 63.2% and the average time was reduced by 33.3%.

/pdf/the-role-of-a-natural-language-conversational-interface-in-4ffvbdzwsr.pdf

The Role of a Natural Language Conversational Interface in Online Sales: A Case Study

We have built a set of integrated AI systems (called conversation machines) to enable transaction processing over the telephone for limited domains like stock trading and banking. The conversation machines integrate the state-of-the-art technologies from computer telephony, continuous speech recognition, natural language processing and humancomputer interaction. Users can interact with these systems using natural language to process simple transactions. We are currently installing a prototype conversation machine at a customer site (a large bank), while continuing research on each of the modules mentioned above and their integration.In this paper, we describe the architecture of conversation machines and explain the design choices related to natural language dialog design, speech recognition errors, and human-computer interaction. We also discuss our experience with the new "market-driven research" methodology currently being tested at our company, of which the conversation machines project is an example. Our experience suggests that with this new methodology we can build integrated natural language dialog systems, even when working with error-prone recognition engines and imperfect grammars, by designing the dialog flow to reduce the likelihood of errors, and to enable quick error recovery. In this process, having a customer allows us to make more realistic design choices.

/pdf/conversation-machines-for-transaction-processing-4465t54h6l.pdf

Conversation machines for transaction processing

In this paper, we propose using standard XML messaging interfaces between components of natural language dialog systems. We describe a stock trading and information access system, where XML is used to encode speech acts, transactions and retrieved information in messages between system components. We use an XML/XSL based unification approach to display personalized, multi-modal system responses. We are proposing the creation of XML standards for all messaging between components of NLP systems. We hope that the use of XML in messaging will promote greater interoperability of both data and code. The working name of the proposed standard messaging language(s) is DSML (“Dialog System Markup Language”.)

/pdf/dsml-a-proposal-for-xml-standards-for-messaging-between-3j1h2p8j6o.pdf

Yiming Ye

Papers

System, method and program product for interactive natural dialog

Comparative evaluation of a natural language dialog based system and a menu driven system for information access: a case study

The Role of a Natural Language Conversational Interface in Online Sales: A Case Study

Conversation machines for transaction processing

DSML: A Proposal for XML Standards for Messaging Between Components of a Natural Language Dialog System