An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

Systems and methods by which voice/data communications may occur in multiple modes/protocols are disclosed. In particular, systems and methods are provided for multiple native mode/protocol voice and data transmissions and receptions with a computing system having a multi-bus structure, including, for example, a TDM bus and a packet bus, and multi-protocol framing engines. Such systems preferably include subsystem functions such as PBX, voice mail and other telephony functions, LAN hub and data router. In preferred embodiments, a TDM bus and a packet bus are intelligently bridged and managed, thereby enabling such multiple mode/protocol voice and data transmissions to be intelligently managed and controlled with a single, integrated system. A computer or other processor includes a local area network controller, which provides routing and hub(s) for one or more packet networks. The computer also is coupled to a buffer/framer, which serves to frame/deframe data to/from the computer from TDM bus. The buffer/framer includes a plurality of framer/deframer engines, supporting, for example, ATM and HDLC framing/deframing. The buffer/framer is coupled to the TDM bus by way of a switch/multiplexer, which includes the capability to intelligently map data traffic between the buffer/framer and the TDM bus to various slots of the TDM frames. Preferably, a DSP pool is coupled to buffer/framer in a manner to provide various signal processing and telecommunications support, such as dial tone generation, DTMF detection and the like. The TDM bus is coupled to a various line/station cards, serving to interface the TDM bus with telephone, facsimiles and other telecommunication devices, and also with a various digital and/or analog WAN network services.

Systems and methods for multiple mode voice and data communications using intelligently bridged TDM and packet buses and methods for performing telephony and data functions using the same

A mobile terminal and a control method thereof, which can obtain an image, are provided. A mobile terminal (100) includes a camera unit (121), a pictogram extraction unit (182) and a controller (180). The camera unit (121) obtains image information corresponding to at least one of a still image and a moving image. The pictogram extraction unit (182) extracts at least one pictogram from the obtained image information. The controller (180) detects information related to the extracted pictogram, and displays the detected information to be overlapped with the obtained image information. In the mobile terminal, the controller (180) includes, as the detected information, at least one of previously recorded information and currently searched information related to the extracted pictogram.

Mobile terminal and control method thereof

A method, system, and apparatus of a touch-based authentication of a mobile device through user generated pattern creation are disclosed. In one embodiment, a method of a mobile device includes recognizing a tactile force on a touch screen without a visual aid as an unlocking gesture, storing the unlocking gesture to a memory of the mobile device, associating another tactile force on the touch screen with the unlocking gesture, and transforming the mobile device from an initial state to an unlocked state based on the association between the another tactile force and the unlocking gesture. The method may include transforming the mobile device to operate as a telephonic communication device in the unlocked state.

Touch-based authentication of a mobile device through user generated pattern creation

A system and method causes a computer to detect and perform actions on structures identified in computer data. The system provides an analyzer server, an application program interface, a user interface and an action processor. The analyzer server receives from an application running concurrently data having recognizable structures, uses a pattern analysis unit, such as a parser or fast string search function, to detect structures in the data, and links relevant actions to the detected structures. The application program interface communicates with the application running concurrently, and transmits relevant information to the user interface. Thus, the user interface can present and enable selection of the detected structures, and upon selection of a detected structure, present the linked candidate actions. Upon selection of an action, the action processor performs the action on the detected structure.

System and method for performing an action on a structure in computer-generated data

Methods and systems for providing graphical user interfaces are described. Overlaid, information-bearing windows whose contents remain unchanged for a predetermined period of time become translucent. The translucency can be graduated so that, over time, if the window's contents remain unchanged, the window becomes more translucent. In addition to visual translucency, windows also have a manipulative translucent quality. Upon reaching a certain level of visual translucency, user input in the region of the window is interpreted as an operation on the underlying objects rather than the contents of the overlaying window.

Graduated visual and manipulative translucency for windows

One of the changes that the World Wide Web has brought to the computing industry is a new way of thinking about documents. Traditionally, documents have been seen as simple streams of characters, like those in a document editor. Applications that manage these documents may do more or less interesting things to the characters, but they rarely attempt to interpret any of the meaning of the document. There's obviously meaning there, but it only becomes apparent when read or otherwise manipulated by a human. In contrast, the Web has brought with it the concept of a document that has been authored in such a way that important bits of information are explicitly identified within the document. This identification exposes some of the meaning of the document, albeit at a fairly low level, so that various kinds of actions primarily \"show me this related document\" are offered to users and made easy for them to carry out.

From documents to objects: an overview of LiveDoc

LiveDoc [6] is an extension to the Macintosh user experience that allows documents to reveal structured information in such a way that it can be readily identified and used to achieve specific actions. Various kinds of recognizers, including contextfree grammars, are used to describe the structures to be found; these structures can be made up of either a single lexical term (either a variable structure like a phone number, or a collection of static strings, like company names) or multiple terms (for instance, a meeting can be defined as a combination of date, time, and venue structures). Small pieces of code can then be associated with each structure to instruct applications to carry out specific user actions on the discovered structuresperhaps to tell a telephony application to \"Dial this phone number.\" These actions can then be offered to users by visually highlighting the discovered structures and attaching pop-up menus to the highlights. (See also [7] for an alternate interface formalism implementing the same notion of structure detection.)

Thomas Bonura

Papers

System and method for performing an action on a structure in computer-generated data

Graduated visual and manipulative translucency for windows

From documents to objects: an overview of LiveDoc

Drop zones: an extension to LiveDoc

WITHDRAWN PATENT AS PER THE LATEST USPTO WITHDRAWN LISTGraduated visual and manipulative translucency for windows

Thomas Bonura

Papers

System and method for performing an action on a structure in computer-generated data

Graduated visual and manipulative translucency for windows

From documents to objects: an overview of LiveDoc

Drop zones: an extension to LiveDoc

***WITHDRAWN PATENT AS PER THE LATEST USPTO WITHDRAWN LIST***Graduated visual and manipulative translucency for windows

WITHDRAWN PATENT AS PER THE LATEST USPTO WITHDRAWN LISTGraduated visual and manipulative translucency for windows