An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

http://cs.uef.fi/sipu/pub/SRE-review-Kinnunen-and-Li.pdf

An overview of text-independent speaker recognition: From features to supervectors

Speech recognition (SR) systems such as Siri or Google Now have become an increasingly popular human-computer interaction method, and have turned various systems into voice controllable systems (VCS). Prior work on attacking VCS shows that the hidden voice commands that are incomprehensible to people can control the systems. Hidden voice commands, though "hidden", are nonetheless audible. In this work, we design a totally inaudible attack, DolphinAttack, that modulates voice commands on ultrasonic carriers (e.g., f > 20 kHz) to achieve inaudibility. By leveraging the nonlinearity of the microphone circuits, the modulated low-frequency audio commands can be successfully demodulated, recovered, and more importantly interpreted by the speech recognition systems. We validated DolphinAttack on popular speech recognition systems, including Siri, Google Now, Samsung S Voice, Huawei HiVoice, Cortana and Alexa. By injecting a sequence of inaudible voice commands, we show a few proof-of-concept attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile. We propose hardware and software defense solutions, and suggest to re-design voice controllable systems to be resilient to inaudible voice command attacks.

https://dl.acm.org/doi/pdf/10.1145/3133956.3134052

DolphinAttack: Inaudible Voice Commands

We propose a novel feature mapping approach that is robust to channel mismatch, additive noise and to some extent, non-linear effects attributed to handset transducers. These adverse effects can distort the short-term distribution of the speech features. Some methods have addressed this issue by conditioning the variance of the distribution, but not to the extent of conforming the speech statistics to a target distribution. The proposed target mapping method warps the distribution of a cepstral feature stream to a standardised distribution over a specified time interval.
We evaluate a number of the enhancement methods for speaker verification, and compare them against a Gaussian target mapping implementation. Results indicate improvements of the warping technique over a number of methods such as Cepstral Mean Subtraction (CMS), modulation spectrum processing, and short-term windowed CMS and variance normalisation.

Feature Warping for Robust Speaker Verification

Voice interfaces are becoming more ubiquitous and are now the primary input method for many devices. We explore in this paper how they can be attacked with hidden voice commands that are unintelligible to human listeners but which are interpreted as commands by devices.

We evaluate these attacks under two different threat models. In the black-box model, an attacker uses the speech recognition system as an opaque oracle. We show that the adversary can produce difficult to understand commands that are effective against existing systems in the black-box model. Under the white-box model, the attacker has full knowledge of the internals of the speech recognition system and uses it to create attack commands that we demonstrate through user testing are not understandable by humans.

We then evaluate several defenses, including notifying the user when a voice command is accepted; a verbal challenge-response protocol; and a machine learning approach that can detect our attacks with 99.8% accuracy.

https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_carlini.pdf

Hidden voice commands

Cepstral domain segmental feature vector normalization for noise robust speech recognition

This paper presents a multi-lingual speaker-independent voice User Interface (UI) that has been implemented for Nokia S60 mobile phones. The paper concentrates on discussing the specific approach used for achieving a multi-lingual and configurable speech recognition and speech synthesis system. The main applications are speaker-independent name dialing and voice commands. The novelty of the applications is that the user does not need to train the voice dialing system but the application reads the user's phonebook and generates the required voice tags automatically. The speaker-independent voice dialing has already been introduced in regions where the language diversity is not so great. The system presented in this paper is the first of its kind to support both speech recognition and speech synthesis in more than 40 languages in embedded devices with strict memory and performance requirements.

Multi-Lingual Speaker-Independent Voice User Interface For Mobile Devices

A speech recognition feature extractor includes a time-to-frequency domain transformer for generating spectral values in the frequency domain from a speech signal; a partitioning means for generating a first set and an additional set of spectral values in the frequency domain; a first feature generator for generating a first group of speech features using the first set of spectral values; a additional feature generator for generating an additional group of speech features using the additional set of spectral values; the feature generators arranged to operate in parallel, an assembler for assembling an output set of speech features from at least one speech feature from the first group of speech features and at least one speech feature from the additional group of speech features, and an anti-aliasing and sampling rate reduction block, where the first and the additional set of spectral values comprise at least one common spectral value.

Speech recognition from overlapping frequency bands with output data reduction

A method and device for providing a mixed audible signal which includes a ringing tone and the identity of a calling party upon receiving the incoming call of the calling party. The incoming call can be a voice call, a call to convey an MMS message, or a call to convey an SMS message. A TTS synthesizer is used to generate sounds representative of the text identifying the caller if the identity can be found in a caller ID database. A MIDI protocol is used to generate a ringing tone of one or more musical note sequences. A combining module is used to integrate the TTS message and the MIDI tone into the mixed audible signal. Whether only the ringing tone or the mixed audible signal is played is selectable based on a use profile of the device.

Text-to-speech and midi ringing tone for communications devices

We outline the main design features of a low complexity speech recognition engine targeted for mobile devices. Although major parts have already been presented, new features and important refinements of the original ideas, which were omitted, are now described. We also show how these techniques can be successfully combined in order to achieve various design targets with minimized impact on the recognition performance.

Olli Viikki

Papers

Cepstral domain segmental feature vector normalization for noise robust speech recognition

Multi-Lingual Speaker-Independent Voice User Interface For Mobile Devices

Speech recognition from overlapping frequency bands with output data reduction

Text-to-speech and midi ringing tone for communications devices

On a practical design of a low complexity speech recognition engine