An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A control server, or similar central processor, manages the distribution of data (including audio and video), voice, and control signals among a plurality of devices connected via a wired and/or wireless communications network. The devices include audio/visual devices (such as, televisions, monitors, PDAs, notepads, notebooks, MP3, portable stereo, etc.) as well as household appliances (such as, lighting, ovens, alarm clocks, etc.). The control server supports video/audio serving, telephony, messaging, file sharing, internetworking, and security. A portable controller allows a user to access and control the network devices from any location within a controlled residential and/or non-residential environment, including its surrounding areas. The controllers are enhanced to support location-awareness and user-awareness functionality.

Method, system, and computer program product for managing controlled residential or non-residential environments

The various methods and devices described herein relate to devices which, in at least certain embodiments, may include one or more sensors for providing data relating to user activity and at least one processor for causing the device to respond based on the user activity which was determined, at least in part, through the sensors. The response by the device may include a change of state of the device, and the response may be automatically performed after the user activity is determined.

Automated Response to and Sensing of User Activity in Portable Devices

A system is described for maintaining synchrony of operations among a plurality of devices having independent clocking arrangements. A task distribution device is to distribute tasks to a synchrony group comprising a plurality of devices to perform tasks distributed by the task distribution device in synchrony. The task distribution device distributes each task to synchrony group members over a network. Each task is associated with a time stamp that indicates a time, relative to a clock maintained by the task distribution device, at which synchrony group members are to execute the task. Each synchrony group member periodically obtains from the task distribution device an indication of current time indicated by its clock, determines a time differential between the task distribution device's clock and its respective clock and determines therefrom a time at which, according to its respective clock, the time stamp indicates that it is to execute the task.

System and method for synchronizing operations among a plurality of independently clocked digital data processing devices

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

Resource management architectures implemented in computer systems to manage resources are described. In one embodiment, a general architecture includes a resource manager and multiple resource providers that support one or more resource consumers such as a system component or application. Each provider is associated with a resource and acts as the manager for the resource when interfacing with the resource manager. The resource manager arbitrates access to the resources provided by the resource providers on behalf of the consumers. A policy manager sets various policies that are used by the resource manager to allocate resources. One policy is a priority-based policy that distinguishes among which applications and/or users have priority over others to use the resources. A resource consumer creates an “activity” at the resource manager and builds one or more “configurations” that describe various sets of preferred resources required to perform the activity. Each resource consumer can specify one or more configurations for each activity. If multiple configurations are specified, the resource consumer can rank them according to preference. This allows the resource consumers to be dynamically changed from one configuration to another as operating conditions change.

Resource manager architecture

Presented is a system and method for determining a user's intent. Specifically, constituents and a topology are derived from the user's expression of intent, which can be stated broadly or stated in specific detail. The intent is expressed verbally, written, or in an XML format. The constituents and topology are resolved into a configuration based upon contexts. The contexts, which include a resource context, a user context, and an application context, includes information about the user's preferences, location, restrictions, device and network availability, and content availability. The configuration is then implemented.

Intent based processing

The present invention relates to a system and method for configuring and managing network devices. The arrival (and departure) of devices on a network can be detected by a monitor. Upon detection, network devices can be simply and dynamically configured with little or no end-user intervention, for instance by automatically loading device drivers and allocating resources for the devices. Furthermore, network devices can be associated with other network devices such as a personal computer to facilitate seamless integration of network devices with a computer operating system.

Simple and dynamic configuration of network devices

Techniques relating to managing multimedia transmissions in terminal services scenarios are described. In one instance, a method sends a user-interface component from a server to a remote client. The method further streams a media component for presentation on the remote client in combination with the user-interface component and wherein the media presentation is tracked but not displayed by the server.

Systems and methods for multimedia remoting over terminal server connections

The present invention analyzes a user's or client application's requirements and searches for a solution based on the end-to-end latency requirements (requested or derived), data formats, control protocols, timing and synchronization, local streaming, and resource availability. The search for a solution is tracked to allow backtracking from the point of no solution. Once a solution is determined, the system translates the solution into requirements for the individual components of the graph.

Dale A. Sather

Papers

Resource manager architecture

Intent based processing

Simple and dynamic configuration of network devices

Systems and methods for multimedia remoting over terminal server connections

Intelligent streaming framework