Audio Hallucination Attacks: Probing the Reliability of Large Audio Language Models
Abstract
Large Audio Language Models (LALMs) achieve strong performance on audio-language tasks; however, their reliability in real-world settings remains underexplored. We introduce Audio Hallucination Attacks (AHA), an attack suite called AHA-Eval, comprising 6.5K QA pairs designed to test whether LALMs genuinely ground their responses in the audio input. AHA targets two attack surfaces: (i) query-based attacks, which exploit question structure to induce hallucinations about absent sounds, and (ii) audio-based attacks, which inject synthetic speech describing non-existent events into the audio stream. Evaluating state-of-the-art LALMs, including Audio Flamingo 3 and Gemini 3 Pro, we observe high attack success rates of 95.35% and 79.65%, respectively, revealing a reliability gap that is hidden by standard benchmark performance. To mitigate this, we propose a 120K QA post-alignment dataset, AHA-Guard, which successfully reduces attack success rates by up to 49%.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces Audio Hallucination Attacks (AHA), a framework for evaluating audio hallucinations in LALMs through innovative query-based and audio-based attack methodologies. This work is significant as it not only identifies critical vulnerabilities in state-of-the-art models but also proposes effective mitigation strategies, paving the way for more reliable audio-language models in real-world applications.
Methodology
The methodology is robust, introducing a novel attack suite (AHA-Eval) that effectively evaluates the reliability of Large Audio Language Models (LALMs) through a systematic approach. The dual focus on query-based and audio-based attacks is particularly insightful, allowing for a comprehensive assessment of model vulnerabilities. The data curation and filtering process is well-structured, ensuring high-quality inputs for the evaluation. The use of LLMs for generating hallucinated sounds and the distinction between explicit and implicit queries are innovative contributions that enhance the depth of the analysis.
Results & evaluation
The experimental setup is thorough, evaluating multiple state-of-the-art LALMs and providing clear metrics for attack success rates. The results demonstrate significant vulnerabilities in these models, with high ASR values indicating a pressing need for improved grounding mechanisms. The comparison of mitigation strategies, particularly the effectiveness of AHA-Guard, is a valuable addition that highlights practical implications for enhancing model reliability.
Reproducibility
The paper provides sufficient detail regarding the experimental setup, including model selection and training procedures, which aids reproducibility. However, the absence of publicly accessible datasets or code limits the ease with which other researchers can replicate the study. Future work should consider releasing the datasets and methodologies used for generating AHA-Eval and AHA-Guard.
Limitations
One limitation is the reliance on specific LALMs for generating hallucinated sounds, which may not generalize across all audio-language models. Additionally, while the evaluation metrics are well-defined, the subjective nature of audio perception may introduce variability in human assessments that are not fully addressed. The paper also does not explore the long-term implications of these vulnerabilities in real-world applications.
Broader impact
The findings have significant implications for the deployment of LALMs in practical applications, particularly in fields such as automated transcription, audio description, and interactive voice response systems. By highlighting the reliability gaps in these models, the research encourages the development of more robust audio grounding techniques, ultimately enhancing the safety and trustworthiness of AI systems in audio processing. The paper introduces Audio Hallucination Attacks (AHA), a framework for evaluating audio hallucinations in LALMs through innovative query-based and audio-based attack methodologies. This work is significant as it not only identifies critical vulnerabilities in state-of-the-art models but also proposes effective mitigation strategies, paving the way for more reliable audio-language models in real-world applications.
Author of this paper? Send a correction.