ARENA: Automated Red-Teaming for Large Audio Language Models
Abstract
Large audio-language models (LALMs) make it possible to interact with language models through speech, music, and environmental sound, but they also introduce a safety surface that is difficult to expose with text-only red-teaming. We study automated audio-grounded red-teaming, where a text query must remain safe in isolation while the joint text-audio input induces harmful target behavior. We propose ARENA, a closed-loop framework that trains a controller on an independent 2,000case text-audio dataset. MD-Judge supplies training rewards and adaptive search feedback, while a separate, non-adaptive Llama Guard 3 evaluator alone labels final outcomes. On 520 held-out AdvBench objectives, ARENA achieves FDR/PSR of 87.9/100.0%, 71.5/96.3%, 68.1/100.0%, and 75.4/98.5% on Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPTAudio, respectively. Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
[The paper presents ARENA, a novel automated red-teaming framework that uses closed-loop feedback and preference optimization to generate effective audio-grounded jailbreaks for Large Audio-Language Models, demonstrating significant vulnerabilities in current safety mechanisms across multiple state-of-the-art models.]
Methodology
The paper proposes ARENA, a closed-loop automated red-teaming framework for Large Audio-Language Models (LALMs). The core methodology involves training a controller LLM to generate text-safe queries paired with audio prompts (either speech or environmental sound) that induce harmful compliance in target models. The training utilizes a 2,000-case seed pool, employing Reward-Weighted Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) based on feedback from an MD-Judge model. During inference, the controller iteratively refines prompts based on judge feedback until a success threshold is met. A key technical distinction is the separation of the search feedback mechanism (MD-Judge) from the final evaluation metric (Llama Guard 3), which is intended to prevent overfitting to the judge. The approach addresses the specific challenge of "audio-grounded" jailbreaks where safety is conditional on the audio modality.
Results & evaluation
The authors evaluate ARENA on four LALMs: Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPT-Audio. They use 520 held-out AdvBench objectives. The results show high Fault Detection Rates (FDR), ranging from 68.1% to 87.9%, significantly outperforming static baselines like AJailBench and JALMBench. The paper includes ablation studies on refinement budgets, target sampling parameters, and audio variant counts, demonstrating that feedback-based refinement and audio variation substantially improve attack discovery. Transferability analysis shows that attacks found on one model can partially transfer to others. The evaluation is comprehensive, covering multiple models and providing detailed failure analysis.
Reproducibility
The paper provides a GitHub link for code. The methodology describes the training data construction (2,000 seeds), the reward shaping formulas, and the DPO setup. However, the specific versions of the TTS models (Piper, TangoFlux) and the exact configuration of the MD-Judge and Llama Guard 3 evaluators are not fully detailed in the text provided, which may hinder exact reproduction. The separation of training and evaluation data is clearly stated, which is good practice.
Limitations
The primary limitation is the reliance on LLM-based judges (MD-Judge and Llama Guard 3) for both training feedback and final evaluation. While the separation is intended to mitigate this, LLM judges are known to have biases and inconsistencies, particularly with audio-grounded reasoning. The paper does not provide human evaluation of the generated jailbreaks or the harmfulness of the responses, which is a significant gap for safety research. Additionally, the effectiveness is limited to the specific audio synthesis models used; different TTS or audio generation models might yield different results. The focus on automated red-teaming also means it may miss nuanced social engineering attacks that require more complex, multi-turn human interaction.
Broader impact
This work has significant implications for the safety of multimodal AI systems. By exposing vulnerabilities in LALMs, it helps developers identify and patch safety gaps before deployment. However, the release of such a powerful red-teaming tool also raises dual-use concerns, as the generated jailbreaks could be misused by malicious actors. The paper responsibly frames this as a safety auditing tool, but the availability of the code and methodology requires careful consideration of access controls. [The paper presents ARENA, a novel automated red-teaming framework that uses closed-loop feedback and preference optimization to generate effective audio-grounded jailbreaks for Large Audio-Language Models, demonstrating significant vulnerabilities in current safety mechanisms across multiple state-of-the-art models.]
Author of this paper? Send a correction.