An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

In one aspect, the present invention facilitates the investigation of networks of criminals, by gathering associations between phone numbers, the names of persons reached at those phone numbers, and voice print data. In another aspect the invention automatically detects phone calls from a prison where the voiceprint of the person called matches the voiceprint of a past inmate. In another aspect the invention detects identity scams in prisons, by monitoring for known voice characteristics of likely imposters on phone calls made by prisoners. In another aspect, the invention automatically does speech-to-text conversion of phone numbers spoken within a predetermined time of detecting data indicative of a three-way call event while monitoring a phone call from a prison inmate. In another aspect, the invention automatically thwarts attempts of prison inmates to use re-dialing services. In another aspect, the invention automatically tags audio data retrieved from a database, by steganographically encoding into the audio data the identity of the official retrieving the audio data.

Multi-party conversation analyzer & logger

A software and/or hardware facility that enables users to associate annotations with text segments contained in digital content. A capture client allows users to create annotations associated with text segments on content being viewed by the user. The annotations are stored in association with the text segments by an annotation server. When a user subsequently views content, text fragments in the viewed content are compared with the stored text segments by the facility. Text segments that are found to match the text fragments are identified by the facility, and the associated annotations displayed to the user on the viewed content by a display client. Because stored annotations are associated with a text segment, rather than the original content or an identifier associated with the original content from which the text segment was identified, annotations are able to be applied to any content that utilizes the text segment in the future.

Capture and display of annotations in paper and electronic documents

A system and method for automatically providing content associated with captured information is described. In some examples, the system receives input by a user, and automatically provides content or links to content associated with the input. In some examples, the system receives input via text entry or by capturing text from a rendered document, such as a printed document, an object, an audio stream, and so on.

Automatically providing content associated with captured information, such as information captured in real-time

An apparatus for collecting data from a plurality of diverse data sources, the diverse data sources generating input data selected from the group including text, audio, and graphics, the diverse data sources selected from the group including real-time and recorded, human and mechanically-generated audio, single-speaker and multispeaker, the apparatus comprising: means for dividing the input data into one or more data segments, the dividing means acting separately on the input data from each of the plurality of diverse data sources, each of the data segments being associated with at least one respective data buffer such that each of the respective data buffers would have the same number of segments given the same data; means for selective processing of the data segments within each of the respective data buffers; and means for distributing at least one of the respective data buffers such that the collected data associated therewith may be used for further processing

Synchronized pattern recognition source data processed by manual or automatic means for creation of shared speaker-dependent speech user profile

An apparatus comprising a session file, session file editor, annotation window, concatenation software and training software. The session file includes one or more audio files and text associated with each audio file segment. The session file editor displays text and provides text selection capability and plays back audio. The annotation window operably associated with the session file editor supports user modification of the selected text, the annotation window saves modified text corresponding to the selected text from the session file editor and audio associated with the modified text. The concatenation software concatenates modified text and audio associated therewith for two or more instances of the selected text. The training software trains a speech user profile using a concatenated file formed by the concatenating software. The session file may have original audio associated with the selected text, wherein the apparatus further comprises software for substituting the modified text for the selected text. In some embodiments, the concatenation software concatenates modified text and audio associated therewith for two or more instances of the selected text. In some embodiments, the training software trains a speech user profile using a concatenated file formed by the concatenating software.

Session file modification with annotation using speech recognition or text to speech

The invention includes a computer implemented method to map an audio file to a verbatim text file. First, a first window may be loaded with a transcribed text file having words. The transcribed text file is associated with the audio file. A second window is loaded with the verbatim text file having a plurality of words. At least one word from the transcribed text file and at least one word from the verbatim text file are selected. The at least one word from the transcribed text file and the at least one word from the verbatim text file is linked. The steps are repeated until all the words in the verbatim text file have been linked.

Speech recognition program mapping tool to align an audio file to verbatim text

A method for locating an audio segment within an audio file comprising (i) providing a first transcribed text file associated with the audio file; (ii) providing a second transcribed text file associated with the audio file; (iii) receiving a user input defining a text segment corresponding to the audio segment to be located; (iv) searching for the text segment in the first transcribed text file; and (v) displaying only those occurrences of the text segment within the first transcribed text file that are also a match to occurrences of the text segment within the second transcribed text file.

Method for locating an audio segment within an audio file

A method to creating a final text from an audio file comprising (a) transcribing the audio file into a transcribed text file using a speech recognition software (1902); (b) loading a first window with the transcribed text file (1904); (c) loading a second window with a previously created text file (1906); (d) comparing the transcribed text file and the previously created file to find differences between the text in the transcribed text file and the text in the previously created text file (1908); (e) correcting the transcribed text file based upon the differences to create the final text (1910). The method may also include searching for the previously created text file.

Michael C. Huttinger

Papers

Synchronized pattern recognition source data processed by manual or automatic means for creation of shared speaker-dependent speech user profile

Session file modification with annotation using speech recognition or text to speech

Speech recognition program mapping tool to align an audio file to verbatim text

Method for locating an audio segment within an audio file

A method for comparing a transcribed text file with a previously created file