An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Embodiments of the invention are directed to a system, method, and software for implementing gestures with touch sensitive devices (such as a touch sensitive display) for managing and editing media files on a computing device or system. Specifically, gestural inputs of a human hand over a touch/proximity sensitive device can be used to control, edit, and manipulate files, such as media files including without limitation graphical files, photo files and video files.

Gestures for controlling, manipulating, and editing of media files using touch sensitive devices

Systems, methods, and devices for interpreting manual swipe gestures as input in connection with touch-sensitive user interfaces that include virtual keyboards are disclosed herein. These allow for a user entering text using the virtual keyboard to perform certain functions using swipes across the key area rather than tapping particular keys. For example, leftward, rightward, upward, and downward swipes can be assigned to inserting a space, backspacing, shifting (as for typing capital letters), and inserting a carriage return and/or new line. Various other mappings are also described. The described techniques can be used in conjunction with a variety of devices, including handheld devices that include touch-screen interfaces, such as desktop computers, tablet computers, notebook computers, handheld computers, personal digital assistants, media players, mobile telephones, and combinations thereof.

Swipe gestures for touch screen keyboards

A portable electronic device displays a plurality of icons (e.g., graphical objects) in a region in a touch-sensitive display; detects a predefined user action, with respect to the touch-sensitive display, for initiating a predefined user interface reconfiguration process; and varies positions of one or more icons in the plurality of icons in response to detecting the predefined user action. The varying includes varying the positions of the one or more icons about respective average positions.

Portable electronic device with interface reconfiguration mode

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

The present invention relates to display of visual content on a client device using server-side rasterization of visual content. Visual content is rendered on a server system, transformed into bitmaps compatible with the display attributes of a client device, and transmitted for display on the client device. The invention allows the server to perform, in effect, as a remote browser for displaying Web pages, e-mail, e-mail attachments, electronic document and forms, database queries and results, drawings, presentations, and images at the client device. The approach is “remote” because the server does the rendering and the client provides the interface; “multi-level” because rendered visual content is represented as a multi-level set of raster representations; and constitutes a “browsing system” because the client and server share data about the source visual content element being browsed, and the client performs a specific browsing function assisted by the server.

Visual content browsing using rasterized representations

A method of navigating within a plurality of bit-maps through a client user interface, comprising the steps of displaying at least a portion of a first one of the bit-maps on the client user interface, receiving a gesture at the client user interface, and in response to the gesture, altering the display by substituting at least a portion of a different one of the bit-maps for at least a portion of the first bit-map

Gesture-based user interface to multi-level and multi-modal sets of bit-maps

Embodiments relate to display of visual content on a client device using server-side rasterization of visual content. According to some embodiments, visual content is rendered on a server system, transformed into bitmaps compatible with the display attributes of a client device, and transmitted for display on the client device. The server system can perform, in effect, as a remote browser for displaying Web pages, e-mail, e-mail attachments, electronic document and forms, database queries and results, drawings, presentations, images at the client device, and so on. The approach can be “remote” because the server does the rendering and the client provides the interface; “multi-level” because rendered visual content is represented as a multi-level set of raster representations; and constitutes a “browsing system” because the client and server share data about the source visual content element being browsed, and the client performs a specific browsing function assisted by the server.

John S. Robotham

Papers

Visual content browsing using rasterized representations

Gesture-based user interface to multi-level and multi-modal sets of bit-maps

Device-Specific Content Versioning

Visual content browsing with zoom and pan features