An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Systems and methods are provided for receiving speech and non-speech communications of natural language questions and/or commands, transcribing the speech and non-speech communications to textual messages, and executing the questions and/or commands. The invention applies context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for one or more users presenting questions or commands across multiple domains. The systems and methods creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context of the speech and non-speech communications and presenting the expected results for a particular question or command.

Systems and methods for responding to natural language speech utterance

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Mobile systems and methods of supporting natural language human-machine interactions

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

There is disclosed an enhanced text entry system which uses word-level analysis to correct inaccuracies automatically in user keystroke entries on reduced-size or virtual keyboards. A method and system are defined which determine one or more alternate textual interpretations of each sequence of inputs detected within a designated auto-correcting region. The actual interaction locations for the keystrokes may occur outside the boundaries of the specific keyboard key regions associated with the actual characters of the word interpretations proposed or offered for selection, where the distance from each interaction location to each corresponding intended character may in general increase with the expected frequency of the intended word in the language or in a particular context. Likewise, in a virtual keyboard system, the keys actuated may differ from the keys actually associated with the letters of the word interpretations. Each such sequence corresponds to a complete word, and the user can easily select the intended word from among the generated interpretations. Additionally, when the system cannot identify a sufficient number of likely word interpretation candidates of the same length as the input sequence, candidates are identified whose initial letters correspond to a likely interpretation of the input sequence.

Virtual keyboard system with automatic correction

Systems and methods for conversational computing and, in particular, to systems and methods for building distributed conversational applications using a Web services-based model wherein speech engines (e.g., speech recognition) and audio I/O systems are programmable services that can be asynchronously programmed by an application using a standard, extensible SERCP (speech engine remote control protocol), to thereby provide scalable and flexible IP-based architectures that enable deployment of the same application or application development environment across a wide range of voice processing platforms and networks/gateways (e.g., PSTN (public switched telephone network), Wireless, Internet, and VoIP (voice over IP)). Systems and methods are further provided for dynamically allocating, assigning, configuring and controlling speech resources such as speech engines, speech pre/post processing systems, audio subsystems, and exchanges between speech engines using SERCP in a web service-based framework.

Universal IP-based and scalable architectures across conversational applications using web services for speech and audio processing resources

A system and method for real-time network communications provides a session identifier (Session ID) as a public key for group communication (20, 22) between clients, and provides a channel identifier (Channel ID) representing a private key for each of a plurality of clients (12). The channel identifier includes client-specific attributes, which function to indicate grouping criteria for the group communication. A dynamic communication link is created over a network between a client (12) and a service (14) based upon the public and private key combination such that group communication is enabled based upon the attributes of the private key and the public key. Communications are translated using a translation service (16) which employs the attributes associated with the private key and the public key combination to provide response information in a designated language to enable multi- lingual real-time communications.

Open architecture based domain dependent real time multi-lingual communication service

A system and method for representing call content in a searchable database includes transcribing call content to text. The call content is projected to vector space, by creating a vector by indexing the call based on the content and determining a similarity of the call to an atomic-class dictionary. The call is classified in a relational database in accordance with the vector.

System and method for management of call data using a vector based model and relational data structure

A method of providing real-time speech correction for a user. The method includes capturing a speech input, performing a real-time recognition of the speech input and analyzing the recognized speech input to identify a possible error in the user's speech.

Real-Time Speech Analysis Method and System

A structure and method for crowdsourcing includes evaluating a metric related to a content to be translated, determining a priority for the content based on the metric related to the content, and queuing the content for crowdsourcing based on the priority determined from the metric.

Andrzej Sakrajda

Papers

Universal IP-based and scalable architectures across conversational applications using web services for speech and audio processing resources

Open architecture based domain dependent real time multi-lingual communication service

System and method for management of call data using a vector based model and relational data structure

Real-Time Speech Analysis Method and System

Selective machine translation with crowdsourcing