Newness and distinctiveness is claimed in the features of ornamentation as shown inside the broken line circle in the accompanying representation.

Display screen or portion thereof with graphical user interface

An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A mobile terminal including a body; a touchscreen provided to a front and extending to side of the body and configured to display content; and a controller configured to detect one side of the body comes into contact with one side of an external terminal, display a first area on the touchscreen corresponding to a contact area of the body and the external terminal and a second area including the content, receive an input of moving the content displayed in the second area to the first area, display the content in the first area, and share the content in the first area with the external terminal.

Mobile terminal and method for controlling the same

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

A mobile terminal and a control method thereof, which can obtain an image, are provided. A mobile terminal (100) includes a camera unit (121), a pictogram extraction unit (182) and a controller (180). The camera unit (121) obtains image information corresponding to at least one of a still image and a moving image. The pictogram extraction unit (182) extracts at least one pictogram from the obtained image information. The controller (180) detects information related to the extracted pictogram, and displays the detected information to be overlapped with the obtained image information. In the mobile terminal, the controller (180) includes, as the detected information, at least one of previously recorded information and currently searched information related to the extracted pictogram.

Mobile terminal and control method thereof

An electronic device with a display and a fingerprint sensor displays a fingerprint enrollment interface and detects, on the fingerprint sensor, a plurality of finger gestures performed with a finger. The device collects fingerprint information from the plurality of finger gestures performed with the finger. After collecting the fingerprint information, the device determines whether the collected fingerprint information is sufficient to enroll a fingerprint of the finger. When the collected fingerprint information for the finger is sufficient to enroll the fingerprint of the finger, the device enrolls the fingerprint of the finger with the device. When the collected fingerprint information for the finger is not sufficient to enroll the fingerprint of the finger, the device displays a message in the fingerprint enrollment interface prompting a user to perform one or more additional finger gestures on the fingerprint sensor with the finger.

Device, method, and graphical user interface for manipulating user interfaces based on fingerprint sensor inputs

A user request is received from a mobile client device, where the user request includes at least a speech input and seeks an informational answer or performance of a task. A failure to provide a satisfactory response to the user request is detected. In response to detection of the failure, information relevant to the user request is crowd-sourced by querying one or more crowd sourcing information sources. One or more answers are received from the crowd sourcing information sources, and the response to the user request is generated based on at least one of the one or more answers received from the one or more crowd sourcing information sources.

Crowd sourcing information to fulfill user requests

Techniques for processing task items are provided. A task item is electronic data that represents a task to be performed, whether manually or automatically. A task item includes one or more details about its corresponding task, such as a description of the task and a location of the task. Specifically, techniques for generating task items, organizing task items, triggering notifications of task items, and consuming task items are described. In one approach, a task item is generated based on input from a user and context of the input. In another approach, different attributes of task items are used to organize the task items intelligently into multiple lists. In another approach, one or more criteria, such as location, are used to determine when to notify a user of a task item. In another approach, actions other than the generation of notification are enabled or automatically performed, actions such as emailing, calling, texting, and searching.

Generating and processing task items that represent tasks to perform

Techniques for triggering an action associated with a task items are provided. A task item associated with a triggering criterion is provided. The triggering criterion requires an occurrence of any of a plurality of communication events. The plurality of communication events includes a telephone call. In some implementations, at least two of the plurality of communication events are of different communication types. In some implementations, the different types are telephone calls, emails, and text messages. The occurrence of a first communication event of the plurality of communication events is detected. It is determined the triggering criterion is satisfied. In response to the determining, a notification associated with the task item is caused to be presented to a user of the electronic device.

Active Transport Based Notifications

An electronic device with one or more processors and memory includes a procedure for training a digital assistant. In some embodiments, the device detects an impasse in a dialogue between the digital assistant and a user including a speech input. During a learning session, the device utilizes a subsequent clarification input from the user to adjust intent inference or task execution associated with the speech input to produce a satisfactory response. In some embodiments, the device identifies a pattern of success or failure associated with an aspect previously used to complete a task and generates a hypothesis regarding a parameter used in speech recognition, intent inference or task execution as a cause for the pattern. Then, the device tests the hypothesis by altering the parameter for a subsequent completion of the task and adopts or rejects the hypothesis based on feedback information collected from the subsequent completion.

Donald W. Pitschel

Papers

Device, method, and graphical user interface for manipulating user interfaces based on fingerprint sensor inputs

Crowd sourcing information to fulfill user requests

Generating and processing task items that represent tasks to perform

Active Transport Based Notifications

Training an at least partial voice command system