An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A touch screen computer executes an application. A method of operating the touch screen computer in response to a user is provided. A virtual input device is provided on the touch screen. The virtual input device comprises a plurality of virtual keys. It is detected that a user has touched the touch screen to nominally activate at least one virtual key, and a behavior of the user with respect to touch is determined. The determined behavior is processed and a predetermined characteristic is associated with the nominally-activated at least one virtual key. A reaction to the nominal activation is determined based at least in part on a result of processing the determined behavior.

Operation of a computer with touch screen interface

Workspaces provided by an object-based user interface appear to share windows and other display objects. Each workspace's data structure includes, for each window in that workspace, a linking data structure called a placement which links to the display system object which provides that window, which may be a display system object in a preexisting window system. The placement also contains display characteristics of the window when displayed in that workspace, such as position and size. Therefore, a display system object can be linked to several workspaces by a placement in each of the workspaces' data structures, and the window it provides to each of those workspaces can have unique display characteristics, yet appear to the user to be the same window or versions of the same window. As a result, the workspaces appear to be sharing a window. Workspaces can also appear to share a window if each workspace's data structure includes data linking to another workspace with a placement to the shared window. The user can invoke a switch between workspaces by selecting a display object called a door, and a back door to the previous workspace is created automatically so that the user is not trapped in a workspace. A display system object providing a window to a workspace being left remains active so that when that workspace is reentered, the window will have the same contents as when it disappeared. Also, the placements of a workspace are updated so that when the workspace is reentered its windows are organized the same as when the user left that workspace. The user can enter an overview display which shows a representation of each workspace and the windows it contains so that the user can navigate to any workspace from the overview.

User interface with multiple workspaces for sharing display system objects

A user can access a common database from a remote communications port, at any qualified location, to generate a map or other positional information which locates selected items of interest, e.g., businesses, stores, architectural sites, and the like. The database contains information representing the items of interest, including, for each of the items of interest, positional coordinates, a geographic vicinity, and a selected category. The positional coordinates discretely locate the vicinity, while the vicinity specifies the exact locations of the items of interest in the selected category. For example, a user in New York can select the display of sporting shops in the area surrounding Chicago O'Hara International Airport selectively. A user can also access a port and display locations of items of interest within the same vicinity as the user and relative to the user's position. The database can be modified from qualified remote locations to change, or add to, the information therein. An advertisement can be tagged to the display or print out as an association with the selected items of interest.

System and methods for remotely accessing a selected group of items of interest from a database

A computer network and a database are used to provide a hardware-independent, dynamic information system in which the information content is entirely user-controlled. Requests are received from individual users of the computer network to electronically publish information, and input is accepted from the individual users. Entries from the users containing the information to be electronically published are automatically collected, classified and stored in the database in searchable and retrievable form. Entries are made freely accessible on the computer network. In response to user requests, the database is searched and entries are retrieved. Entries are served to users in a hardware-independent page description language. The entries are password protected, allowing users to retrieve and update entries by supplying a correct password. Preferably, the process is entirely automated with any necessary billing being performed by secure, on-line credit card processing. The user making a database entry has complete control of that entry both at the time the entry is made and in the future after the entry has been made. The entry, when served to a client, is transformed on-the-fly to the page description language. Where the page description language is HTML and the computer network is the World Wide Web, the entry may function as a “mini” homepage for the user that made the entry. Provision is made for graphics and other kinds of content besides text, taking advantage of the content-rich nature of the Web.

Automated on-line information service and directory, particularly for the world wide Web

A technique is disclosed which facilitates the selection by a user of a graphic object on a display screen. Selection is by means of a pointing cursor which is moved about the display screen by means of a pointing device having a select button. When the pointing cursor is brought within a predetermined distance of a graphic object, the point on the graphic object closest to the pointing cursor is highlighted to show the user that that graphic object would be selected if the select button were pressed. If the user presses the select button when a point on a graphic object is highlighted, the entire graphic object is highlighted to indicate that it has been selected.

Graphic highlight adjacent a pointing cursor

Improvements in an application composite editor for compound documents containing not only text but also graphics and tables facilitate the manipulation of object sets in the formatting algorithm. The editor works with a page layout philosophy wherein data objects reside on the page and data resides in the data objects. All pages reside within a document object, and some data objects may have additional objects within them. Objects are data-specific entities that the user can manipulate on the page. All objects exist within a specified boundary on the page, and this boundary is defined as an object set boundary. Object sets may be moved into positions on the page such that more than one object set is occupying a single displayable area on the page. Such an arrangement of objects creates a structure called a superblock. A superblock is any displayable area containing two or more object sets positioned so that the object sets overlap one another, reside side-by-side or extend above or below one another. A text object set may not be overlapped by any other object set. Although the superblock is itself a complex structure, the creation of this structure by the editor greatly simplifies integration of different data types on the page for the user and allows the user to manipulate a group of object sets within a single displayable area on the page with relative ease. Moreover, formatting of the document is facilitated by the editor since the superblock is treated as an object set without taking into consideration the complexity inside the superblock structure except when a page end decision must be made.

In-line dynamic editor for mixed object document

A method for tailoring a master document to obtain a special document. The master document is made up of data objects and portions of the data objects. Included in the portions are pointers to corresponding data objects. The portions are windowed and displayed. Selection of a window will result in a view port being opened through the window into a corresponding data object. The data object can then be scrolled to obtain a desired portion to be included in the special document. As each desired portion is obtained, it is stored and becomes part of the special document.

Method for generating a document utilizing a plurality of windows associated with different data objects

Improvements in an application composite eidtor for compound documents containing not only text but also graphics and tables facilitate the manipulation of object sets in the formatting algorithm The editor works with a page layout philosophy wherein data objects reside on the page and data resides in the data objects All pages reside within a document object, and some data objects may have additional objects within them Objects are data-specific entities that the user can manipulate on the page All objects exist within a specified boundary on the page, and this boundary is defined as an object set boundary Object sets may be moved into positions on the page such that more than one object set is occupying a single displayable are on the page Such an arrangement of objects creates a structure called a superblock A superblock is any displayable area containing two or more object sets positioned so that the object sets overlap one another, reside side-by-side or extend above or below one another A text object set may not be overlapped by any other object set Although the superblock is itself a complex structure, the creation of this structure by the editor greatly simplifies integration of different data types on the page for the user and allows the user to manipulate a group of object sets within a single displayable area on the page with relative ease Moreover, formatting of the document is facilitated by the editor since the superblock is treated as an object set without taking into consideration the complexity inside the superblock structure except when a page end decision must be made

Editing of a superblock data structure

A method of, and system for, merging a portion of one document into another and providing for current viewing and on line editing During preparation of a document, a paragraph from another document can be included in the document being prepared by referencing the other document and paragraph The result of referencing is inclusion of the paragraph, a view of the document being prepared, and a document including the included paragraph which is dynamically editable If the paragraph is edited, an operator decision is to be made as to whether the other document is to be updated according to the editing

Rex Allen Mccaskill

Papers

Graphic highlight adjacent a pointing cursor

In-line dynamic editor for mixed object document

Method for generating a document utilizing a plurality of windows associated with different data objects

Editing of a superblock data structure

Merging of documents