Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis
Abstract
Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains unclear how much jailbreak capability can arise from matched-text variation in speech delivery rather than from lexical rewriting or broader style transfer. We study this question by holding transcript content fixed and varying six speech-delivery presets whose acoustic attributes may co-vary. We present PJ-Break, a black-box evaluation protocol with presets targeting arousal, authority, and speaking rate, together with AdvAudio-Prosody, a 600-sample benchmark with acoustically verified attributes. On the exact post-QC Qwen2-Audio panel, the Q=1 Panic (38/95), Anger (35/95), and Fast (32/95) presets are all well above Neutral (4/95). The fixed six-query pool covers 44/95 Qwen2-Audio seeds and 15/95 GPT-4o seeds and exceeds a matched-budget StyleBreak reimplementation (27/95) on Qwen2-Audio. A same-voice pool excluding the confounded Commanding condition still reaches 40/95, and a retained-panel ablation shows emotional-delivery audio alone (44/95) is far more effective than emotional text alone (11/95). Exploratory surrogate diagnostics and pilot mitigation observations are secondary, non-core analyses. Overall, matched-text speech delivery should be treated as a first-class factor in Audio LLM safety evaluation
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper makes a valuable contribution to multimodal safety by empirically demonstrating that prosodic variations in speech delivery can significantly compromise the safety of Audio LLMs, independent of lexical content. Its controlled methodology and clear empirical findings provide a necessary foundation for future research into robust, prosody-aware safety mechanisms for spoken AI assistants.
Methodology
The paper proposes a controlled experimental framework, PJ-Break, to isolate the impact of prosodic variations (arousal, authority, speaking rate) on the safety/jailbreak capabilities of Audio Large Language Models (Audio LLMs). By holding the lexical transcript constant and varying only the speech delivery via TTS presets, the authors address a specific and under-explored gap in multimodal safety: the influence of paralinguistic cues. The methodology is rigorous in its control of variables, utilizing a fixed TTS stack and explicit acoustic verification. The inclusion of an ablation study comparing emotional text vs. emotional audio is a strong methodological choice that directly supports the core hypothesis. The use of surrogate model diagnostics (activation patching) to hypothesize mechanisms, while clearly labeled as exploratory, adds depth to the analysis.
Results & evaluation
The evaluation is conducted on Qwen2-Audio and GPT-4o, using a benchmark of 600 samples (AdvAudio-Prosody). The results are compelling: prosodic jailbreaks significantly outperform neutral audio and even matched-budget style-transfer baselines (StyleBreak). The finding that "Panic" and "Anger" presets achieve high success rates (38/95 and 35/95 respectively) compared to Neutral (4/95) is a significant empirical contribution. The same-voice sensitivity analysis strengthens the claim that the effect is due to prosody rather than voice identity. However, the reliance on TTS-generated speech limits the ecological validity, although the authors do provide small-scale human-speech and OTA checks to mitigate this concern. The statistical significance (McNemar test) adds robustness to the main comparison.
Reproducibility
The paper provides detailed implementation specifications, including the TTS engine (Azure Neural TTS), specific voices used, and the definition of the six presets. The acoustic verification metrics ($F_0$, RMS, etc.) are standard and reproducible. However, the dataset and code are explicitly *not* released due to safety concerns regarding the generation of harmful audio jailbreaks. This is a valid ethical stance for this domain but limits independent verification and extension. The description of the evaluation protocol and judge ensemble is sufficiently detailed for a competent researcher to replicate the core experiments, assuming access to the same models and TTS services.
Limitations
The primary limitation is the reliance on TTS, which may not fully capture the nuances of human emotional speech or the specific acoustic artifacts that might arise in real-world over-the-air attacks. The "Commanding" condition is partially confounded by voice identity (different speaker), which the authors acknowledge but does not fully eliminate the confound. The study is limited to English and a single TTS provider, limiting generalizability to other languages and synthesis technologies. Furthermore, the surrogate analysis is restricted to one model (Qwen2-Audio), and the mechanistic insights (refusal direction) are hypothesis-generating rather than definitive proof of causality in closed-source models. The pilot mitigation results are preliminary and not robust.
Broader impact
This paper has significant implications for the safety evaluation of multimodal AI systems. It demonstrates that current safety filters, which often focus on text or content, may be bypassed by manipulating the *delivery* of the content. This necessitates a shift in safety research to include prosodic and paralinguistic analysis in red-teaming protocols. It also highlights potential vulnerabilities in crisis-response or assistance-oriented models that might be triggered by high-arousal speech. The work encourages the development of prosody-aware safety mechanisms and more robust multimodal safety benchmarks. The paper makes a valuable contribution to multimodal safety by empirically demonstrating that prosodic variations in speech delivery can significantly compromise the safety of Audio LLMs, independent of lexical content. Its controlled methodology and clear empirical findings provide a necessary foundation for future research into robust, prosody-aware safety mechanisms for spoken AI assistants.
Author of this paper? Send a correction.