An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Voice trigger for a digital assistant

A device can be configured to receive speech input from a user. The speech input can include a command for accessing a restricted feature of the device. The speech input can be compared to a voiceprint (e.g., text-independent voiceprint) of the user's voice to authenticate the user to the device. Responsive to successful authentication of the user to the device, the user is allowed access to the restricted feature without the user having to perform additional authentication steps or speaking the command again. If the user is not successfully authenticated to the device, additional authentication steps can be request by the device (e.g., request a password).

Device access using voice authentication

A computer-implemented method comprising: providing a transcription of an utterance for presentation in an area of a display of a computing device; receiving a user selection of at least one word of the transcription of the utterance; in response to receiving the user selection of the at least one word of the transcription of the utterance, presenting at the display of the computing device one or more of: (i) one or more alternate words for the user-selected at least one word of the transcription of the utterance and (ii) a remove control to remove the user-selected at least one word of the transcription of the utterance from the transcription of the utterance; receiving a user selection of the remove control or an alternate word from among the one or more alternate words; and updating the transcription of the utterance presented in the area of the display of the computing device based at least on the user selection. A system and computer program are also described.

Word-level correction of speech input

Systems and processes are disclosed for handling a multi-part voice command for a virtual assistant. Speech input can be received from a user that includes multiple actionable commands within a single utterance. A text string can be generated from the speech input using a speech transcription process. The text string can be parsed into multiple candidate substrings based on domain keywords, imperative verbs, predetermined substring lengths, or the like. For each candidate substring, a probability can be determined indicating whether the candidate substring corresponds to an actionable command. Such probabilities can be determined based on semantic coherence, similarity to user request templates, querying services to determine manageability, or the like. If the probabilities exceed a threshold, the user intent of each substring can be determined, processes associated with the user intents can be executed, and an acknowledgment can be provided to the user.

Multi-command single utterance input method

Features are disclosed for managing the use of speech recognition models and data in automated speech recognition systems. Models and data may be retrieved asynchronously and used as they are received or after an utterance is initially processed with more general or different models. Once received, the models and statistics can be cached. Statistics needed to update models and data may also be retrieved asynchronously so that it may be used to update the models and data as it becomes available. The updated models and data may be immediately used to re-process an utterance, or saved for use in processing subsequently received utterances. User interactions with the automated speech recognition system may be tracked in order to predict when a user is likely to utilize the system. Models and data may be pre-cached based on such predictions.

Speech model retrieval in distributed speech recognition systems

Embodiments of systems and methods are described for inverting transformations of signals due to room acoustics In some implementations, a transformation of a calibration signal from a particular location in a room may be determined From this transformation, an inverse transformation may be determined and the inverse transformation may be applied to a speech signal received from a similar location

Transformation inversion to reduce the effect of room acoustics

A method for transcribing a spoken communication includes acts of receiving a spoken first communication from a first sender to a first recipient, obtaining information relating to a second communication, which is different from the first communication, from a second sender to a second recipient, using the obtained information to obtain a language model, and using the language model to transcribe the spoken first communication.

Methods and systems for obtaining language models for transcribing communications

In a distributed automated speech recognition (ASR) system, speech models may be employed on a local device to allow the local device to process frequently spoken utterances while passing other utterances to a remote device for processing. Upon receiving an audio signal, the local device compares the audio signal to the speech models of the frequently spoken utterances to determine whether the audio signal matches one of the speech models. When the audio signal matches one of the speech models, the local device processes the utterance, for example by executing a command. When the audio signal does not match one of the speech models, the local device transmits the audio signal to a second device for ASR processing. This reduces latency and the amount of audio signals that are sent to the second device for ASR processing.

Local speech recognition of frequent utterances

An unsupervised acoustic modeling service for speech recognition is disclosed. A computing device may be present in a listening zone, such as a household or office, and may receive an audio signal that may include speech. In some instances, the speech is not directed to the computing device. Speech recognition results may be generated from the audio signal, using an acoustic model, and the results used to update the acoustic model. For example, the acoustic model may be updated to reflect a particular speaker's pronunciation of certain sound units, such as phonemes. By using speech that is not necessarily directed to the computing device, more data may be available for updating an acoustic model.

Jeffrey C. O'Neill

Papers

Speech model retrieval in distributed speech recognition systems

Transformation inversion to reduce the effect of room acoustics

Methods and systems for obtaining language models for transcribing communications

Local speech recognition of frequent utterances

Unsupervised acoustic model training