An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

The various methods and devices described herein relate to devices which, in at least certain embodiments, may include one or more sensors for providing data relating to user activity and at least one processor for causing the device to respond based on the user activity which was determined, at least in part, through the sensors. The response by the device may include a change of state of the device, and the response may be automatically performed after the user activity is determined.

Automated Response to and Sensing of User Activity in Portable Devices

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user's intent and to reduce the number of candidate interpretations of the user's input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Using context information to facilitate processing of commands in a virtual assistant

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

Method and apparatus for building an intelligent automated assistant

Among other things, techniques and systems are disclosed for implementing contextual voice commands. On a device, a data item in a first context is displayed. On the device, a physical input selecting the displayed data item in the first context is received. On the device, a voice input that relates the selected data item to an operation in a second context is received. The operation is performed on the selected data item in the second context.

Contextual voice commands

A method and system used to determine the similarity between an input speech data and a sample speech data is provided. First, the input speech data is segmented into a plurality of input speech frames and the sample speech data is segmented into a plurality of sample speech frames. Then, the input speech frames and the sample speech frames are used to build a matching matrix, wherein the matching matrix comprises the distance values between each of the input speech frames and each of the sample speech frames. Next, the distance values are used to calculate a matching score. Finally, the similarity between the input speech data and the sample speech data is determined according to this matching score.

Method and system for matching speech data

The sequential minimal optimization (SMO) algorithm has been extensively employed to train the support vector machine (SVM). This work presents an efficient application specific integrated circuit chip design for sequential minimal optimization. This chip is implemented as an intellectual property core, suitable for use in an SVM-based recognition system on a chip. The proposed SMO chip was tested and found to be fully functional, using a prototype system based on the Altera DE2 board with a Cyclone II 2C70 field-programmable gate array.

VLSI Design of an SVM Learning Core on Sequential Minimal Optimization Algorithm

This work presents an unsupervised speaker change detection algorithm based on support vector machines (SVM) to detect speaker change (SC) in a speech stream. The proposed algorithm is called the SVM training misclassification rate (STMR). The STMR can identify SCs with less speech data collection, making it capable of detecting speaker segments with short duration. According to experiments on the NIST Rich Transcription 2005 Spring Evaluation (RT-05S) corpus, the STMR has a missed detection rate of only 19.67 percent.

Unsupervised speaker change detection using SVM training misclassification rate

Embedded speaker identification system is a popular research, but most of current systems can not provide fast training ability Because of the low computational ability in the embedded environment, a large amount of waiting time usually makes the human-machine interface not friendly This paper presents a hardware and software (HW/SW) co-design solution for fast-trainable speaker identification system Fast training ability makes this embedded speaker identification system possess high flexibility and enhances the convenience to a wide range of real-world applications The proposed system consists of a training phase and a multiclass identification phase The sequential minimal optimization (SMO) training algorithm occupies the heaviest computational load and is realized as a dedicated VLSI module, ie, the hardware component The other processes such as speech preprocess, speech feature extraction, and SVM voting strategy are implemented by software Moreover, a data-packed mechanism is presented to improve the bandwidth utilization Compared with the embedded C code based on ARM processor, our system reduces 90% of the training time and achieves 899% identification rate with the NIST 2010 speaker recognition database The proposed system was tested and found to be fully functional working on a Socle CDK prototype system with an AMBA based Xilinx FPGA and an ARM926EJ processor

Hardware/software co-design for fast-trainable speaker identification system based on SMO

This paper presents a human-robot interactive system for senior companion based on cloud computing infrastructure. The proposed senior companion robot system (SCRS) is designed based on cloud computing network. In the server side, two cloud services are proposed, 1) the web-based user remote management service (WURMS) for remote robot control; 2) the robotic multimodal interactive computation services (RMICS) for providing the human-robot operation interfaces including the speech/sound recognition, speaker identification, face identification, sound source estimation and text to speech (TTS). In the robot client side, the behavior model is designed to use WURMS and RMICS services. In the experiments, two robots called “Robert” and “Davinci” are designed to evaluate the SCRS's capability. With using only low-cost and low-power CPUs (Intel Atom N450), both of the two robots can still work wirelessly for real-time human-robot interaction. Finally, we design five senior companion scenarios, and the experimental average MOS (Mean Opinion Score) is 4.16.

Po-Chuan Lin

Papers

Method and system for matching speech data

VLSI Design of an SVM Learning Core on Sequential Minimal Optimization Algorithm

Unsupervised speaker change detection using SVM training misclassification rate

Hardware/software co-design for fast-trainable speaker identification system based on SMO

Human-robot interaction based on cloud computing infrastructure for senior companion