An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Systems and methods are provided for receiving speech and non-speech communications of natural language questions and/or commands, transcribing the speech and non-speech communications to textual messages, and executing the questions and/or commands. The invention applies context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for one or more users presenting questions or commands across multiple domains. The systems and methods creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context of the speech and non-speech communications and presenting the expected results for a particular question or command.

Systems and methods for responding to natural language speech utterance

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Mobile systems and methods of supporting natural language human-machine interactions

A mobile terminal and a control method thereof, which can obtain an image, are provided. A mobile terminal (100) includes a camera unit (121), a pictogram extraction unit (182) and a controller (180). The camera unit (121) obtains image information corresponding to at least one of a still image and a moving image. The pictogram extraction unit (182) extracts at least one pictogram from the obtained image information. The controller (180) detects information related to the extracted pictogram, and displays the detected information to be overlapped with the obtained image information. In the mobile terminal, the controller (180) includes, as the detected information, at least one of previously recorded information and currently searched information related to the extracted pictogram.

Mobile terminal and control method thereof

A conversational, natural language voice user interface may provide an integrated voice navigation services environment. The voice user interface may enable a user to make natural language requests relating to various navigation services, and further, may interact with the user in a cooperative, conversational dialogue to resolve the requests. Through dynamic awareness of context, available sources of information, domain knowledge, user behavior and preferences, and external systems and devices, among other things, the voice user interface may provide an integrated environment in which the user can speak conversationally, using natural language, to issue queries, commands, or other requests relating to the navigation services provided in the environment.

System and method for providing a natural language voice user interface in an integrated voice navigation services environment

Techniques are provided for determining collaborative notes and automatically recognizing speech, handwriting and other type of information. Domain and optional actor/speaker information associated with the support information is determined. An initial automatic speech recognition model is determined based on the domain and/or actor information. The domain and/or actor/speaker language model is used to recognize text in the speech information associated with the support information. Presentation support information such as slides, speaker notes and the like are determined. The semantic overlap between the support information and the salient non-function words in the recognized text and collaborative user feedback information are used to determine relevancy scores for the recognized text. Grammaticality, well formedness, self referential integrity and other features are used to determine correctness scores. Suggested collaborative notes are displayed in the user interface based on the salient non-function words. User actions in the user interface determine feedback signals. Recognition models such as automatic speech recognition, handwriting recognition are determined based on the feedback signals and the correctness and relevance scores.

Systems and methods for collaborative note-taking

A structural natural language index is created by segmenting documents within a repository into text portions and extracting named entity, co-reference, lexical entries, structural-semantic relationships, speaker attribution and meronymic derived features. A constituent structure is determined that contains the constituent elements and ordering information sufficient to reconstruct the text portion. A functional structure of the text portions is determined. A set of characterizing predicative triples are formed from the functional structure by applying linearization transfer rules. The constituent structure, the characterizing predicative triples and the derived features are combined to form a canonical form of the text portion. Each canonical form is added to the structural natural language index. A retrieved question is classified to determine question type and a corresponding canonical form for the question is generated. The entries in the structural natural language index are searched for entries matching the canonical form of the question and relevant to the question type. The characterizing predicative triples are used in conjunction with a generation grammar to create an answer. If the generation fails, some or all of the constituent structure of the matching entry is returned as the answer.

Systems and methods for structural indexing of natural language text

Methods, systems and computer readable media for finding documents in a data store that match a natural language query submitted by a user are provided. The documents and queries are matched by determining that words within the query have the same relationship to each other as the same words in the document. Documents are semantically analyzed and words in the document are indexed along with the role the word plays in a sentence. The initial semantic role may be generalized using a role hierarchy and stored in the index along with the original role. A similar analysis may be used with the search query to find words used in the same role in both the query and the document.

Indexing role hierarchies for words in a search index

Computer-readable media and computer systems for conducting semantic processes to facilitate navigation of search results that include sets of tuples representing facts associated with content of documents in response to queries for information. Content of documents is accessed and semantic structures are derived by distilling linguistic representations from the content. Groups of two or more related words, called tuples, are extracted from the documents or the semantic structures. Tuples can be stored at a tuple index. Representations of the relational tuples are displayed in addition to documents retrieved in response to a query.

Browsing knowledge on the basis of semantic relations

A physical key is used to propose an association between a guest device and user information and services. Contact-based or contact-less connectors are used to establish the proposed association between the physical key and the guest device. The proposed association then communicated to the dynamic authentication system over a first communication channel. The dynamic authentication system determines a user confirmation over a second communications channel based on a user device and previously determined associations between users, user devices and the physical key. The guest device is then authenticated for access to information and/or services associated with the user. The information retrieved from and/or transmitted to the user's personal information repository is optionally protected using various transformations. Optional session identifiers supported on the physical key and/or the user device, ensure the protected information is inaccessible when the physical key is removed and/or the predetermined association with the user device is deleted.

Giovanni L Thione

Papers

Systems and methods for collaborative note-taking

Systems and methods for structural indexing of natural language text

Indexing role hierarchies for words in a search index

Browsing knowledge on the basis of semantic relations

Systems and methods for dynamic authentication using physical keys