An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

Aspects of the present invention relate to a compact optics module for a head mounted display including a DLP image source comprising a polarized light source providing illumination light, a flat reflective polarizer to reflect the illumination light toward the DLP image source and transmit image light, wherein the illumination light is provided at an angle to the DLP, a quarter wave retarder to change the polarization state of the illumination light as it is reflected by the DLP image source thereby providing image light, at least one light trap to absorb a portion of off-state image light, a lens element to provide a viewable field of view of on-state image light in an image displayed by the DLP image source, and a combiner to reflect the on-state image light toward a user's eye and provide a see-through view of a surrounding environment.

See-through computer display systems

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

An electronic device may capture a voice command from a user. The electronic device may store contextual information about the state of the electronic device when the voice command is received. The electronic device may transmit the voice command and the contextual information to computing equipment such as a desktop computer or a remote server. The computing equipment may perform a speech recognition operation on the voice command and may process the contextual information. The computing equipment may respond to the voice command. The computing equipment may also transmit information to the electronic device that allows the electronic device to respond to the voice command.

Electronic Devices with Voice Command and Contextual Data Processing Capabilities

This invention is based on being able to locate a voice server, temporarily allocate it, send it the audio of you saying 'When is flight 683 due to arrive?', getting the results of what you said back in the browser, and deallocating the voice server for use by the next person talking into their browser. Voice channels and IVR ports are initially set up by a switch and the IVR using conventional audio protocols. The Voice channels are not initially connected to the client. The switch handles the allocation and deallocation of IVR voice channels without having to communication further with the IVR. A user indicates (usually by pressing a PTT button) to the client device that he wishes to initiate a voice interaction during an X+V session. This translates to a request on the CTRL channel to synchronise the XHTML and VXML forms which the embodiment uses as a trigger for the VXML browser to execute a conversational turn. The multiplexer intercepts this control command and connects the virtual voice circuit between the device and an existing open but unattached voice port. The virtual circuit is connected without having to set up an RTP channel. The CTRL signal is then forwarded to the interaction manager so that the conversation can take place. At the end of the conversation the virtual circuit is disconnected.

Method and apparatus for multimodal voice and web services

A method for enhancing voice interactions within a portable multimodal computing device using visual messages. A multimodal interface can be provided that includes an audio interface and a visual interface. A speech input can then be received and a voice recognition task can be performed upon at least a portion of the speech input. At least one message within the multimodal interface can be visually presented, wherein the message is a prompt for the speech input and/or a confirmation of the speech input.

Method of enhancing voice interactions using visual messages

An apparatus and method as shown automatically selects a video stream of a video-conference for transmission or display. The apparatus and method includes a receiving step for receiving video and audio streams over a network from participants in a video-conference. Each of the audio streams each has amplitude data. A suppressing step suppresses some of either the first or second video stream based on the amplitude data of the corresponding audio stream. The video stream or streams that are not suppressed are either displayed on a display screen of a participant of the video conference or transmitted to other terminals for display on display screens.

Method and apparatus for automatic chat room source selection based on filtered audio input amplitude of associated data streams

A method of integrating conversational speech into a multimodal, Web-based processing model can include speech recognizing a user spoken utterance directed to a voice-enabled field of a multimodal markup language document presented within a browser. A statistical grammar can be used to determine a recognition result. The method further can include providing the recognition result to the browser, receiving, within a natural language understanding (NLU) system, the recognition result from the browser, and semantically processing the recognition result to determine a meaning. Accordingly, a next programmatic action to be performed can be selected according to the meaning.

Integrating conversational speech into Web browsers

Decompression of video segments from a sequence of differential frames is done by selected scaling of frame resolution and color depth. A frame header indicates the computational complexity of decompression of a frame, allowing selection of scales for output resolution and color depth. Decompression proceeds by retrieving a frame from the compressed video stream in elementary units. An elementary unit is characterized by types, including an unchanged type, a homogeneous type, a pattern type and a predetermined pattern type. For a retrieved elementary unit of the unchanged type, an output pointer to a display buffer is moved by an elementary unit scaled by the output resolution scale. For a retrieved elementary unit of the homogeneous type, a color retrieved from the compressed video stream is applied to an area in the display buffer corresponding to an elementary unit scaled by the output resolution scale. For a retrieved elementary unit of the predetermined pattern type, a pattern from a table of patterns is retrieved using an index from the compressed video segment as an address. Two colors for the pattern are also retrieved and applied to the pattern.

Leslie R. Wilson

Papers

Method and apparatus for multimodal voice and web services

Method of enhancing voice interactions using visual messages

Method and apparatus for automatic chat room source selection based on filtered audio input amplitude of associated data streams

Integrating conversational speech into Web browsers

Scalable digital video decompressor