An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

In one aspect, the present invention facilitates the investigation of networks of criminals, by gathering associations between phone numbers, the names of persons reached at those phone numbers, and voice print data. In another aspect the invention automatically detects phone calls from a prison where the voiceprint of the person called matches the voiceprint of a past inmate. In another aspect the invention detects identity scams in prisons, by monitoring for known voice characteristics of likely imposters on phone calls made by prisoners. In another aspect, the invention automatically does speech-to-text conversion of phone numbers spoken within a predetermined time of detecting data indicative of a three-way call event while monitoring a phone call from a prison inmate. In another aspect, the invention automatically thwarts attempts of prison inmates to use re-dialing services. In another aspect, the invention automatically tags audio data retrieved from a database, by steganographically encoding into the audio data the identity of the official retrieving the audio data.

Multi-party conversation analyzer & logger

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

The method includes automatically, without user input and without regard to whether a digital assistant application has been separately invoked by a user, determining that the electronic device is in a vehicle. In some implementations, determining that the electronic device is in a vehicle comprises detecting that the electronic device is in communication with the vehicle (e.g., via a wired or wireless communication techniques and/or protocols). The method also includes, responsive to the determining, invoking a listening mode of a virtual assistant implemented by the electronic device. In some implementations, the method also includes limiting the ability of a user to view visual output presented by the electronic device, provide typed input to the electronic device, and the like.

Automatically adapting user interfaces for hands-free interaction

An audio-feature analyzer automatically detects points of change in audio signals to be processed. A central processing unit (CPU) obtains point-of-change information indicating positions of the points of change in the audio signals, and the point-of-change information is recorded on a data storage device. The CPU identifies point-of-change information in accordance with an instruction input by a user via a key operation unit, and audio data corresponding to the point-of-change information identified is located so that processing such as playback of audio data to be processed can be started therefrom.

Audio signal processing apparatus and audio signal processing method

PROBLEM TO BE SOLVED: To quickly find and utilize an objective portion of audio data which are processing objects without perplexing a user. SOLUTION: An audio feature analysis section 143 automatically detects a changed point of audio signals that are processing objects, a CPU101 obtains changed point information which indicates the position of the changed point on the audio signals and records the changed point information in a data recorder 111. The CPU101 specifies the changed point information corresponding to the instruction input received from the user through a key operation section 121, conducts the positioning of the audio data corresponding to the specified changed point information and conducts processes such as play back or the like of audio data that are the processing objects from the position. COPYRIGHT: (C)2005,JPO&NCIPI

Audio signal processor, audio signal processing method, and audio signal processing program

A recording and reproducing device including sound collecting means for collecting external sound, and outputting sound information corresponding to a result of sound collection, photographing means for photographing an outside, and outputting image information corresponding to a result of photographing, recording means for converting the sound information to files in message units and recording the files on a predetermined recording medium, and recording the image information on the recording medium in association with a corresponding file of sound information in response to a predetermined first external operation, display means for displaying images based on image information associated with respective files of sound information, and reproducing means for reproducing a file of sound information associated with an image selected by a predetermined second external operation among the images displayed by the display means. The recording means sets, for each file of sound information, one of images based on image information associated with the file as a representative image of the file.

Recording and reproducing device and recording and reproducing method

PROBLEM TO BE SOLVED: To retrieve a recorded voice file with a voice input to find out a target location, and to automatically start the reproduction of voice data from the target location, and to attach a callable mark to the found location when the found location is necessary. SOLUTION: An IC recorder etc. having a recording part which records a voice from a microphone as data and a voice reproducing part which reproduces the recorded voice data is provided with a retrieving part for retrieving the specific part of the recorded data by using a voice inputted from the microphone as a retrieval key, means (S301 to S304) for, when a target location is found out by retrieval, starting clocking for 5 seconds, and for performing reproduction output, and for, when an ENTER key is depressed within 5 seconds, adding a mark to the location matched with the retrieval key and a mark addition completing means (S305). The mark is added to the matched location, so that the mark location can be easily called, and that the reproduction of voice data can be automatically performed from the matched location. COPYRIGHT: (C)2005,JPO&NCIPI

Voice data retrieval apparatus

Disclosed are a portable recording/reproducing apparatus for recording predefined information onto a recording medium and reproducing the recorded information, and a method for controlling the apparatus. The apparatus includes a recording section to collect an external sound and output audio information in accordance with the sound collection results; a shooting section to shoot the outside and output motion picture information in accordance with shooting results; a display section to display a motion picture in accordance with the motion picture information; a retention section to retain the shooting section as desired in a ready-for-shooting state or in a storage state in which shooting cannot be performed; a detection section to detect whether or not the shooting section is retained in the ready-for-shooting state by the retention section; and a control section, which, when the detection section detects at the beginning of audio information recording that the shooting section is retained in the ready-for-shooting state, exercises control so that the motion picture based on the motion picture information appears on the display section. The apparatus can record a still picture based on a motion picture while recording audio information.

Satoshi Mihara

Papers

Audio signal processing apparatus and audio signal processing method

Audio signal processor, audio signal processing method, and audio signal processing program

Recording and reproducing device and recording and reproducing method

Voice data retrieval apparatus

Recording/reproducing apparatus and recording/reproducing apparatus control method