An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

An electronic device may capture a voice command from a user. The electronic device may store contextual information about the state of the electronic device when the voice command is received. The electronic device may transmit the voice command and the contextual information to computing equipment such as a desktop computer or a remote server. The computing equipment may perform a speech recognition operation on the voice command and may process the contextual information. The computing equipment may respond to the voice command. The computing equipment may also transmit information to the electronic device that allows the electronic device to respond to the voice command.

Electronic Devices with Voice Command and Contextual Data Processing Capabilities

The method includes automatically, without user input and without regard to whether a digital assistant application has been separately invoked by a user, determining that the electronic device is in a vehicle. In some implementations, determining that the electronic device is in a vehicle comprises detecting that the electronic device is in communication with the vehicle (e.g., via a wired or wireless communication techniques and/or protocols). The method also includes, responsive to the determining, invoking a listening mode of a virtual assistant implemented by the electronic device. In some implementations, the method also includes limiting the ability of a user to view visual output presented by the electronic device, provide typed input to the electronic device, and the like.

Automatically adapting user interfaces for hands-free interaction

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Voice trigger for a digital assistant

The techniques disclosed herein allow a user to synchronize the playing and displaying of digital content on an electronic device. The device may render a first portion of digital content so it may be displayed. The device may also play a segment of the digital content as audio using text to speech software. The device may also render a second portion of digital content for display depending on whether the position of the last word read is greater than the last position in the first portion of digital content.

Synchronizing the playing and displaying of digital content

A user's progress through a digital work may be presented using a progress bar. The progress bar may display a visualization of consumed text of the digital work with respect to unconsumed text of the digital work. An expanded portion of the progress bar may also be displayed. A reading rate associated with the user's consumption of the digital work and a consumption time of the digital work may be presented.

Progress indication for a digital work

A digital media item, such as an electronic book (eBook), may include testing content. The testing content may include questions about the content of the digital media item. When is user is viewing the digital media item on an electronic device, such as an eBook reader, the user may be allowed to select whether the testing content is displayed. The user may also be allowed to select a particular mode of testing, such as automatic testing, selective testing, etc. If the user chooses to display the testing content, the user may also be allowed to provide answers to the testing questions.

Testing within digital media items

Zone-associated objects enable easier consumption of electronic media items that are presented to a user on a screen of a computing device. Zone-associated objects enable displaying electronic content on the screen of the computing device, the screen having multiple zones. Each object presented on the screen may be associated with one or more of the zones. The user may then select one of the multiple zones and, in response, may be allowed to select objects associated with that zone. In some instances, the user may then select one or more of these zone-associated objects to perform the action defined by the object.

Zone-Associated Objects

A user's progress through a digital work may be presented using a progress gauge One or more content dividers may be presented dividing the progress gauge into segments Each segment of the progress gauge may correspond to a section of the digital work, and may visually represent an amount of content in the corresponding section of the digital work, relative to other sections of the digital work A collection of digital works available for access by a computing device may be displayed A progress gauge may be presented in association with one or more of the digital works in the collection of available digital works

Laurent An Minh Nguyen

Papers

Synchronizing the playing and displaying of digital content

Progress indication for a digital work

Testing within digital media items

Zone-Associated Objects

Relative progress and event indicators