Voice assistants (VA) have become prevalent on a wide range of personal devices such as smartphones and smart speakers. As companies build voice assistants with extra functionalities, attacks that trick a voice assistant into performing malicious behaviors can pose a significant threat to a user’s security, privacy, and even safety. However, the diverse attacks and stand-alone defenses in the literature often lack a systematic perspective, making it challenging for designers to properly identify, understand, and mitigate the security threats against voice assistants. To overcome this problem, this article provides a thorough survey of the attacks and countermeasures for voice assistants. We systematize a broad category of relevant but seemingly unrelated attacks by the vulnerable system components and attack methods, and categorize existing countermeasures based on the defensive strategies from a system designer’s perspective. To assist designers in planning defense based on their demands, we provide a qualitative comparison of existing countermeasures by the implementation cost, usability, and security and propose practical suggestions. We envision this work can help build more reliability into voice assistants and promote research in this fast-evolving area.

A Survey on Voice Assistant Security: Attacks and Countermeasures

Automatic speech recognition (ASR) is an essential technology used in commercial products nowadays. However, the underlying deep learning models used in ASR systems are vulnerable to adversarial examples (AEs), which are generated by applying small or imperceptible perturbations to audio to fool these models. Recently, universal adversarial perturbations (UAPs) have attracted much research interest. UAPs used to generate audio AEs are not limited to a specific input audio signal. Instead, given a generic audio signal, audio AEs can be generated by directly applying UAPs. This paper presents a method of generating UAPs based on a targeted phrase. To the best of our knowledge, our proposed method of generating UAPs is the first to successfully attack ASR models with connectionist temporal classification (CTC) loss. In addition to generating UAPs, we empirically show that the UAPs can be considered as signals that are transcribed as the target phrase. We also show that the UAPs themselves preserve temporal dependency, such that the audio AEs generated using these UAPs also preserved temporal dependency.

Targeted Universal Adversarial Perturbations for Automatic Speech Recognition

Recent research has shown that Deep Neural Networks (DNNs) are highly vulnerable to adversarial samples, which are highly transferable and can be used to attack other unknown black-box models. To improve the transferability of adversarial samples, several feature-based adversarial attack methods have been proposed to disrupt neuron activation in the middle layers. However, current state-of-the-art feature-based attack methods typically require additional computation costs for estimating the importance of neurons. To address this challenge, we propose a Singular Value Decomposition (SVD)-based feature-level attack method. Our approach is inspired by the discovery that eigenvectors associated with the larger singular values decomposed from the middle layer features exhibit superior generalization and attention properties. Specifically, we conduct the attack by retaining the decomposed Top-1 singular value-associated feature for computing the output logits, which are then combined with the original logits to optimize adversarial examples. Our extensive experimental results verify the effectiveness of our proposed method, which can be easily integrated into various baselines to significantly enhance the transferability of adversarial samples for disturbing normally trained CNNs and advanced defense strategies. The source code of this study is available at https://github.com/WJJLL/SVD-SSA

Boosting Adversarial Transferability via Fusing Logits of Top-1 Decomposed Feature

In the physical world, deep neural networks (DNNs) are impacted by light and shadow, which can have a significant effect on their performance. While stickers have traditionally been used as perturbations in most physical attacks, their perturbations can often be easily detected. To address this, some studies have explored the use of light-based perturbations, such as lasers or projectors, to generate more subtle perturbations, which are artificial rather than natural. In this study, we introduce a novel light-based attack called the adversarial neon beam (AdvNB), which utilizes common neon beams to create a natural black-box physical attack. Our approach is evaluated on three key criteria: effectiveness, stealthiness, and robustness. Quantitative results obtained in simulated environments demonstrate the effectiveness of the proposed method, and in physical scenarios, we achieve an attack success rate of 81.82%, surpassing the baseline. By using common neon beams as perturbations, we enhance the stealthiness of the proposed attack, enabling physical samples to appear more natural. Moreover, we validate the robustness of our approach by successfully attacking advanced DNNs with a success rate of over 75% in all cases. We also discuss defense strategies against the AdvNB attack and put forward other light-based physical attacks.

Adversarial Neon Beam: A Light-based Physical Attack to DNNs

Audio adversarial detection through classification score on speech recognition systems

Adversarial attacks have been widely recognized as the security vulnerability of deep neural networks, especially in deep automatic speech recognition (ASR) systems. The advanced detection methods against adversarial attacks mainly focus on pre-processing the input audio to alleviate the threat of adversarial noise. Although these methods could detect some simplex adversarial attacks, they fail to handle robust complex attacks especially when the attacker knows the detection details. In this paper, we propose a unified adversarial detection framework for detecting adaptive audio adversarial examples, which combines noise padding with sound reverberation. Specifically, a well-designed adaptive artificial utterances generator is proposed to balance the design complexity, such that the artificial utterances (speech with reverberation) are efficiently determined to reduce the false positive rate and false negative rate of detection results. Moreover, to destroy the continuity of the adversarial noise, we develop a novel multi-noise padding strategy, which implants the Gaussian noises in the silent fragments of the input speech by the voice activity detector. Furthermore, our proposed method can effectively tackle the robust adaptive attacks in an adaptive learning manner. Importantly, the conceived system is easily embedded into any ASR models without requiring additional retraining or modification. The experimental results show that our method consistently outperforms the state-of-the-art audio defense methods, even for the adaptive and robust attacks.

A Unified Framework for Detecting Audio Adversarial Examples

Many studies on deep neural networks have shown very promising results for most image recognition tasks. However, these networks can often be fooled by adversarial examples that simply add small but powerful distortions to the original input. Recent works have demonstrated the vulnerability of deep learning systems to adversarial examples, but most such works directly manipulate and attack the digital images for a specific classifier only, and cannot attack the physical images in real world. In this paper, we propose the multi-sample ensemble method (MSEM) and most-likely ensemble method (MLEM) to generate adversarial attacks that successfully fool the classifier for images in both the digital and real worlds. The proposed adaptive norm algorithm can craft faster and smaller perturbation than other state-of-the-art attack methods. Besides, the proposed MLEM extended with weighted objective function can generate robust adversarial attacks that can mislead multiple classifiers (Inception-v3, Inception-v4, Resnet-v2, Ince-res-v2) simultaneously for physical images in real world. Compared with other methods, experiments show that our adversarial attack methods not only can achieve higher success rates but also can survive in the multi-model defense tests.

Xia Du

Papers

A Unified Framework for Detecting Audio Adversarial Examples

Adversarial Image Attacks Using Multi-Sample and Most-Likely Ensemble Methods