An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

The method includes automatically, without user input and without regard to whether a digital assistant application has been separately invoked by a user, determining that the electronic device is in a vehicle. In some implementations, determining that the electronic device is in a vehicle comprises detecting that the electronic device is in communication with the vehicle (e.g., via a wired or wireless communication techniques and/or protocols). The method also includes, responsive to the determining, invoking a listening mode of a virtual assistant implemented by the electronic device. In some implementations, the method also includes limiting the ability of a user to view visual output presented by the electronic device, provide typed input to the electronic device, and the like.

Automatically adapting user interfaces for hands-free interaction

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Voice trigger for a digital assistant

A portable device, such as a cellular telephone, engages the user in a challenge-response sequence that is based on recognition of the user's utterance and also upon verification of the user's speech patterns or voiceprint The challenge-response protocol presents the user with an unexpected challenge word, which the user is then requested to respond to The system maintains a secure data store of challenge words which it adapts and augments as the user makes use of the portable device The portable device provides the user with a convenient, single access point through which he or she can authenticate with a variety of disparate secure devices ranging from door locks, ATM machines, financial institutions and third-party business associates

System and method for portable authentication

A new speaker provides speech from which comparison snippets are extracted. The comparison snippets are compared with initial snippets stored in a recorded snippet database that is associated with a concatenative synthesizer. The comparison of the snippets to the initial snippets produces required sound units. A greedy selection algorithm is performed with the required sound units for identifying the smallest subset of the input text that contains all of the text for the new speaker to read. The new speaker then reads the optimally selected text and sound units are extracted from the human speech such that the recorded snippet database is modified and the speech synthesized adopts the voice quality and characteristics of the new speaker.

Text selection and recording by feedback and adaptation for development of personalized text-to-speech systems

An MITalk-based real-time system for unrestricted English, Japanese, and Chinese text-to-speech synthesis has been developed with high observed intelligibility (94% for word recognition in English and 70.8% for 119 Japanese monosyllables). The approach in making the system multilingual has been simple yet effective: in developing the Japanese and Chinese systems, the authors have used the hardware and the program structure and tools of the system originally developed for English, but have otherwise kept the development of the different languages separate. In addition to substantial improvements and modifications of the original MITalk, two major innovations in the system are (1) the method for controlling the glottal pulse and (2) the Phoenix development tools, which combine a powerful and flexible parameter editor with programs for acoustic analysis. >

http://ieeexplore.ieee.org/iel2/805/6677/00266410.pdf

A multilingual text-to-speech system

A prosody modification system for use in text-to-speech includes an input receiving a sequence of prosodic data vectors Pn, measured at time Tn, which samples a sound waveform. A prosody data warping module directly derives new prosodic data vectors Qn from the original data vectors Pn using a function, which is controlled by warping parameters A0, . . . Ak, which avoids round-off errors in deriving quantized values, which has derivatives with respect to A0, . . . Ak, Pn, and Tn that are continuous, and which has sufficiently high complexity to model intentional prosody of the sound waveform, and sufficiently low complexity to avoid modeling micro-prosody of the sound waveform. The smoothness and simplicity of the function ensure that micro-prosodic perturbations and errors in measurement of Tn are transferred directly to the output Qn. The errors are thus reversed during re-synthesis and therefore eliminated, resulting in micro-prosodic perturbations being preserved during re-synthesis.

Prosody generation for text-to-speech synthesis based on micro-prosodic data

A MITalk-based [l] real-time system for unres tricted English, Japanese and Chinese text-tc-speech synthesis has been developed with high observed intelligibility (94% for word recognition in English and 70.8% for 119 Japanese monosyllables). Our approach in making the system multi-lingual has been simple yet effective: in developing the Japanese and Chinese systems, we have used the hardware and the program structure and tools of the system originally developed for English, but have otherwise kept the development of the different languages separate. In addition to substantial improvements and modifications of the original MITalk, two major innovations in our system are (1) the method for controlling the glottal pulse and (2) the Phoenix develop ment tools, which combine a powerful and flexible parameter editor with programs for acoustic analysis.

Steven Pearson

Papers

System and method for portable authentication

Text selection and recording by feedback and adaptation for development of personalized text-to-speech systems

A multilingual text-to-speech system

Prosody generation for text-to-speech synthesis based on micro-prosodic data

A Multi-lingual Text-to-Speech System