An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

A method, system, and computer program product for enabling messaging systems to use alternative message delivery mechanisms. In existing instant messaging systems, messages cannot be delivered to an intended recipient unless the recipient is currently logged on to an instant messaging system. According to the present invention, users may register one or more alternative message delivery mechanisms (such as pagers, cell phones, etc.) through which they are available as an alternative to an instant messaging system. Constraints may optionally be added to these registered alternatives, such as specifying a limitation on days of the week and/or hours of the day when a particular alternative may be used. Providing that an alternative messaging mechanism is registered, and any constraints for this alternative are satisfied, in the first preferred embodiment the extended IMS will select a suitable alternative and deliver the message. In a second and third preferred embodiment, the sender of the “instant” message will be informed of the available alternative(s). The sender may then choose to have his message delivered even though the intended receiver is not currently logged on to an instant messaging system. Fourth and fifth embodiments enable a message to be delivered to a recipient using a registered mechanism (that may include an IMS), and provide for determining availability of a selected user. Using these techniques, the real-time awareness of potential message receivers in instant messaging systems is extended.

Technique for enabling messaging systems to use alternative message delivery mechanisms

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

An electronic device may capture a voice command from a user. The electronic device may store contextual information about the state of the electronic device when the voice command is received. The electronic device may transmit the voice command and the contextual information to computing equipment such as a desktop computer or a remote server. The computing equipment may perform a speech recognition operation on the voice command and may process the contextual information. The computing equipment may respond to the voice command. The computing equipment may also transmit information to the electronic device that allows the electronic device to respond to the voice command.

Electronic Devices with Voice Command and Contextual Data Processing Capabilities

A method, system, and machine-readable medium are provided for utilizing a network repository having stored voice font data. A request for a response, including the voice font data stored in the network repository; is received via a network. The voice font data stored in the network repository is accessed. The response, including the voice font data, is sent via the network.

Network repository for voice fonts

A method and system for automatic speech recognition are disclosed. The method comprises receiving speech from a user, the speech including at least one speech error, increasing the probabilities of closely related words to the at least one speech error and processing the received speech using the increased probabilities. A corpora of data having common words that are mis-stated is used to identify and increase the probabilities of related words. The method applies to at least the automatic speech recognition module and the spoken language understanding module.

System and method for augmenting spoken language understanding by correcting common errors in linguistic performance

Disclosed herein are systems, methods, and computer readable-media for performing an audible human verification. The method includes determining that a human verification is needed, presenting an audible challenge to a user which exploits a known issue with automatic speech recognition processes, receiving a response to the audible challenge, and verifying that a human provided the response. The known issue with automatic speech recognition processes can be recognition of a non-word, in which case the user can be asked to spell the recognized non-word. The known issue with automatic speech recognition processes can be differentiation of simultaneous input for multiple audio streams. Multiple audio streams contained in the audible challenge can be provided monaurally. Verifying that a human provided the response can include confirming the contents of one of the multiple audio streams. Audible human verification can be performed in combination with visual human verification.

System and method for auditory captchas

A messaging system with a remote message recording device is provided. To compose and send a message to an intended recipient, the sending party remotely records a spoken message on a recording device. When recording the message, the sending party adds a predefined spoken header before the actual subject matter of the message and also adds a predefined spoken closure comment at the end of the message. The spoken header includes identification information for the intended recipient of the message. To transmit the message to the intended recipient, the sending party downloads the recorded message to a message transmission system. The transmission system includes speech recognition software. The speech recognition software interprets the identification information for the intended recipient contained in the spoken header of the message. The transmission system utilizes this identification information for the intended recipient to access a database that contains a routing access code for the intended recipient. The transmission system transmits the message to the intended recipient by utilizing the routing access code obtained from the database.

Messaging system with remote messaging recording device where the message is routed based on the spoken name of the recipient

A system and method is provided for rapidly generating a new spoken dialog application. In one embodiment, a user experience person labels the transcribed data (e.g., 3000 utterances) using a set of interactive tools. The labeled data is then stored in a processed data database. During the labeling process, the user experience person not only groups utterances in various call type categories, but also flags (e.g., 100-200) specific utterances as positive and negative examples for use in an annotation guide. The labeled data in the processed data database can also be used to generate an initial natural language understanding (NLU) model.

Steven H. Lewis

Papers

Network repository for voice fonts

System and method for augmenting spoken language understanding by correcting common errors in linguistic performance

System and method for auditory captchas

Messaging system with remote messaging recording device where the message is routed based on the spoken name of the recipient

System and method for automatic generation of a natural language understanding model