Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs
Abstract
Speech-driven large language models (LLMs) are increasingly accessed through speech interfaces, introducing new security risks via open acoustic channels. We present Sirens' Whisper (SWhisper), the first practical framework for covert prompt-based attacks against speech-driven LLMs under realistic black-box conditions using commodity hardware. SWhisper enables robust, inaudible delivery of arbitrary target baseband audio-including long and structured prompts-on commodity devices by encoding it into near-ultrasound waveforms that demodulate faithfully after acoustic transmission and microphone nonlinearity. This is achieved through a simple yet effective approach to modeling nonlinear channel characteristics across devices and environments, combined with lightweight channel-inversion pre-compensation. Building on this high-fidelity covert channel, we design a voice-aware jailbreak generation method that ensures intelligibility, brevity, and transferability under speech-driven interfaces. Experiments across both commercial and open-source speech-driven LLMs demonstrate strong black-box effectiveness. On commercial models, SWhisper achieves up to 0.94 non-refusal (NR) and 0.925 specific-convincing (SC). A controlled user study further shows that the injected jailbreak audio is perceptually indistinguishable from background-only playback for human listeners. Although jailbreaks serve as a case study, the underlying covert acoustic channel enables a broader class of high-fidelity prompt-injection and commandexecution attacks.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of Sirens' Whisper (SWhisper), a novel framework for covert prompt-based attacks against speech-driven LLMs, demonstrating significant advancements in the security landscape of machine learning applications. The technical contributions are substantial, addressing a critical gap in the understanding of vulnerabilities in speech-driven AI systems and providing a practical methodology for exploiting these vulnerabilities.
Methodology
The methodology presented in this paper is innovative, leveraging near-ultrasonic waveforms to create a covert channel for prompt injection into speech-driven LLMs. The authors effectively model nonlinear channel characteristics and utilize lightweight channel-inversion pre-compensation, which is a significant contribution to the field of security in machine learning. The voice-aware jailbreak generation method is particularly noteworthy as it ensures the intelligibility and transferability of the injected prompts, making the approach practical and effective under realistic conditions.
Results & evaluation
The experiments conducted demonstrate strong black-box effectiveness across both commercial and open-source speech-driven LLMs. The reported metrics, such as non-refusal (NR) and specific-convincing (SC) rates, indicate a high level of success in the covert prompt delivery. Additionally, the user study validating the perceptual indistinguishability of the injected audio from background noise adds credibility to the findings. However, the paper could benefit from a more extensive evaluation across diverse environments and device types.
Reproducibility
The paper lacks detailed implementation specifics, such as code availability or datasets used for training and evaluation, which may hinder reproducibility. While the methodology is sound, providing access to the experimental setup would enhance the ability of other researchers to replicate the results.
Limitations
One limitation of the study is the reliance on specific hardware and environmental conditions that may not generalize across all devices or settings. Additionally, the focus on jailbreaks as a case study may limit the perceived applicability of the covert channel to other types of attacks. The paper could also address potential countermeasures that could be employed against such attacks.
Broader impact
The implications of this research are significant, as it highlights a new vector for security vulnerabilities in speech-driven LLMs, which are becoming increasingly prevalent in various applications. The findings could inform the development of more robust security measures in AI systems that utilize speech interfaces, ultimately contributing to safer deployment in real-world scenarios. The broader class of attacks enabled by the covert acoustic channel also raises important questions about the security of voice-activated systems. The main contribution of this paper is the introduction of Sirens' Whisper (SWhisper), a novel framework for covert prompt-based attacks against speech-driven LLMs, demonstrating significant advancements in the security landscape of machine learning applications. The technical contributions are substantial, addressing a critical gap in the understanding of vulnerabilities in speech-driven AI systems and providing a practical methodology for exploiting these vulnerabilities.
Author of this paper? Send a correction.