An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

The method includes automatically, without user input and without regard to whether a digital assistant application has been separately invoked by a user, determining that the electronic device is in a vehicle. In some implementations, determining that the electronic device is in a vehicle comprises detecting that the electronic device is in communication with the vehicle (e.g., via a wired or wireless communication techniques and/or protocols). The method also includes, responsive to the determining, invoking a listening mode of a virtual assistant implemented by the electronic device. In some implementations, the method also includes limiting the ability of a user to view visual output presented by the electronic device, provide typed input to the electronic device, and the like.

Automatically adapting user interfaces for hands-free interaction

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Voice trigger for a digital assistant

Disclosed herein is a segmentation method, which divides an input audio stream into segments containing different homogeneous signals. The main objective of this method is localization of segments with stationary properties. This method seeks all no-stationary points or intervals in the audio stream and creates a list of segments. The obtained list of segments can be used as an input data for the following procedures, such as classification, speech/music/noise attribution and so on. The proposed segmentation method is based on the analysis of audio signal statistical features variation and comprises three main stages: stage of first-grade characteristics calculation, stage of second-grade characteristics calculation and stage of decision-making.

Method of segmenting an audio stream

The present invention relates to method and system for distinguishing speech from music in a digital audio signal in real time. A method for distinguishing speech from music in a digital audio signal in real time for the sound segments that have been segmented from an input signal of the digital sound processing systems by means of a segmentation unit on the base of homogeneity of their properties, comprises the steps of: (a) framing an input signal into sequence of overlapped frames by a windowing function; (b) calculating frame spectrum for every frame by FFT transform; (c) calculating segment harmony measure on base of frame spectrum sequence; (d) calculating segment noise measure on base of the frame spectrum sequence; (e) calculating segment tail measure on base of the frame spectrum sequence; (f) calculating segment drag out measure on base of the frame spectrum sequence; (g) calculating segment rhythm measure on base of the frame spectrum sequence; and (h) making the distinguishing decision based on characteristics calculated.

Method and system for distinguishing speech from music in a digital audio signal in real time

A method of filtering digital audio data with short delay according to the present invention comprises the steps of: (a) buffering input source digital audio data; (b) calculating digital data that are being filtered from the buffered input source digital audio data; and (c) outputting a portion of the filtered digital data. The source digital audio data are sequence of digital samples. Buffer contents are shifted to arrange some of input samples in a buffer in the step (a). The step (b) comprises the steps of: (b-1) calculating a correlation matrix; (b-2) decomposing the correlation matrix; (b-3) calculating a filter matrix; and (b-4) calculating an approximation of the filtered digital data. In the step (b-2), the correlation matrix is represented by multiplication of three matrices where a first matrix is a matrix composed by normalized eigenvectors of the correlation matrix, a second matrix is a diagonal matrix in which eigenvalues of the correlation matrix are arranged on a principal diagonal and zeros are arranged in other positions, and a third matrix is a transposed matrix of the normalized eigenvectors of the correlation matrix.

Method of filtering noise of source digital data

PURPOSE: A method and system for identifying music/voice of a digital audio signal in real time are provided to identify music/voice in real-time audio data processing. CONSTITUTION: An input signal is framed into a sequence of overlapped frames according to a window function. Frame spectrum is calculated for all of the frames according to Fast Fourier Transform. The quantity of measurement of segment harmony(30) is calculated on the basis of the frame spectrum sequence. The quantity of measurement of segment noise(40) is calculated on the basis of the frame spectrum sequence. The quantity of measurement of segment tail(50) is calculated on the basis of the frame spectrum sequence. The quantity of measurement of segment drag out(60) is calculated on the basis of the frame spectrum sequence. The quantity of measurement of segment rhythm(70) is calculated on the basis of the frame spectrum sequence. Identification is determined on the basis of the calculated characteristics.

/ Method and system for real-time music/speech discrimination in digital audio signals

PURPOSE: A method for filtering noise of primitive digital data is provided to remove noise in a signal such as audio streams broadcasted or recorded using predetermined media with relatively simple integration circuit. CONSTITUTION: Inputted primitive digital audio data is buffered. Filtered digital data is calculated for the buffered digital audio data. A portion of the filtered digital data is output. The inputted primitive digital data is buffered in such a manner that buffer contents are moved to arrange input samples in a buffer. The step of calculating the filtered digital data includes a step of calculating correlation matrix, a step of decomposing the correlation matrix, a step of calculating filter matrix in filter matrix calculator(4), and a step of approximately calculating the filtered digital data.

Anatoli I. Tikhotsky

Papers

Method of segmenting an audio stream

Method and system for distinguishing speech from music in a digital audio signal in real time

Method of filtering noise of source digital data

/ Method and system for real-time music/speech discrimination in digital audio signals

Method of noise filtering of source digital data