Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard
Abstract
Recent progress in large language models (LLMs) has enabled understanding of both speech and non-speech audio, but exposing new safety risks emerging from complex audio inputs that are inadequately handled by current safeguards. We introduce SACRED-Bench (Speech-Audio Composition for RED-teaming) to evaluate the robustness of LLMs under complex audio-based attacks. Unlike existing perturbation-based methods that rely on noise optimization or white-box access, SACRED-Bench exploits speech-audio composition mechanisms. SACRED-Bench adopts three mechanisms: (a) speech overlap and multi-speaker dialogue, which embeds harmful prompts beneath or alongside benign speech; (b) speech-audio mixture, which imply unsafe intent via non-speech audio alongside benign speech or audio; and (c) diverse spoken instruction formats (open-ended QA, yes/no) that evade text-only filters. Experiments show that, even Gemini 2.5 Pro, the state-of-the-art proprietary LLM, still exhibits 66% attack success rate in SACRED-Bench test set, exposing vulnerabilities under cross-modal, speech-audio composition attacks. To bridge this gap, we propose SALMONN-Guard, a safeguard LLM that jointly inspects speech, audio, and text for safety judgments, reducing attack success down to 20%. Our results highlight the need for audio-aware defenses for the safety of multimodal LLMs. The benchmark and SALMONN-Guard checkpoints can be found at https://huggingface.co/datasets/tsinghua-ee/SACRED-Bench. Warning: this paper includes examples that may be offensive or harmful.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a novel approach to evaluating and mitigating vulnerabilities in multimodal LLMs through innovative audio-based attacks and defenses. Its contributions are significant for advancing the safety and robustness of AI systems in handling complex audio inputs.
Methodology
The methodology is innovative, introducing SACRED-Bench, a benchmark that exploits the complexities of audio inputs for red-teaming multimodal LLMs. The three attack mechanisms—speech overlap, speech-audio mixture, and diverse spoken instruction formats—are well-conceived and demonstrate a clear understanding of the vulnerabilities in existing models. The construction of the attacks is methodical, leveraging both semantic and acoustic layers to obscure harmful content, which is a significant advancement over previous perturbation-based methods.
Results & evaluation
The experiments are robust, with a comprehensive evaluation of multiple state-of-the-art models, including proprietary and open-source LLMs. The reported attack success rates provide compelling evidence of the vulnerabilities in these models, and the introduction of SALMONN-Guard as a mitigation strategy is well-supported by experimental results showing a significant reduction in attack success rates. The use of a diverse dataset for training and testing enhances the reliability of the findings.
Reproducibility
The paper provides sufficient detail regarding the experimental setup, including the datasets used and the evaluation metrics. However, the lack of a publicly available implementation or code repository may hinder full reproducibility. The authors mention using Hugging Face for dataset availability, but further details on the model training and evaluation processes would strengthen reproducibility.
Limitations
One limitation is the potential ethical concerns surrounding the creation and use of harmful audio content for testing purposes. Additionally, while the proposed SALMONN-Guard shows promise, its effectiveness in real-world applications remains to be fully validated. The reliance on specific models for evaluation may also limit the generalizability of the findings to other LLMs not included in the study.
Broader impact
The research has significant implications for the safety and robustness of multimodal LLMs, particularly in applications where audio inputs are prevalent. The introduction of SACRED-Bench and SALMONN-Guard could lead to improved safety measures in AI systems, enhancing their reliability in real-world scenarios. This work emphasizes the importance of developing comprehensive defenses against emerging threats in AI, particularly as models become more capable of processing complex inputs. The paper presents a novel approach to evaluating and mitigating vulnerabilities in multimodal LLMs through innovative audio-based attacks and defenses. Its contributions are significant for advancing the safety and robustness of AI systems in handling complex audio inputs.
Author of this paper? Send a correction.