SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision
Abstract
Audio deepfake detection (ADD) must remain effective when new spoofing attacks emerge after deployment. Emerging audio language model (ALM)-based ADD methods are built on predefined supervision from ground-truth labels or verified forensic rationales. However, this paradigm overlooks an ALM's own mistakes, which indicate where targeted supervision is most needed. To this end, we first introduce evolving spoofing environments for ALM-based ADD, where a new attack becomes dominant while previously observed attacks persist. Motivated by the above learning-from-mistakes perspective, we further propose SE-ADD, a self-evolving framework that iteratively adapts an ALM via low-rank adaptation (LoRA) using mistake-driven supervision built from its verdicts and self-generated forensic cues. All training samples receive direct authenticity supervision, while misclassified ones receive additional cue-augmented supervision. As verdicts and cues are regenerated by the updated ALM, the resulting supervision evolves accordingly. Experiments on two ALMs demonstrate the effectiveness of SE-ADD in generalizing to unseen attacks, reducing the equal error rate (EER) from $36.72\%$ to $7.52\%$ for Qwen2-Audio and from $19.93\%$ to $3.97\%$ for MOSS-Audio.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces SE-ADD, a self-evolving framework for audio deepfake detection that leverages mistake-driven supervision and self-generated forensic cues to adapt ALMs to evolving spoofing environments, demonstrating significant improvements in generalization to unseen attacks.
Methodology
The paper proposes SE-ADD, a framework for self-evolving audio deepfake detection (ADD) using Audio Language Models (ALMs). The core methodological contribution is a "mistake-driven supervision" loop where the model generates its own forensic cues and uses its classification errors to construct targeted training data. This data is used to update LoRA adapters iteratively. The framework introduces a novel "evolving spoofing environment" setup using a Norton-Bass activity process to simulate shifting attack distributions. While the idea of self-improvement via self-generated data is not entirely new in LLMs, applying it specifically to audio deepfake detection with a structured evolving environment and forensic cue generation is a solid technical contribution. The distinction between "direct" supervision for all samples and "cue-augmented" supervision for mistakes is a reasonable heuristic for focusing learning on hard examples.
Results & evaluation
Experiments are conducted on two ALMs (Qwen2-Audio and MOSS-Audio) using the ASVspoof 2019 LA dataset. The evaluation covers within-environment adaptation and held-out generalization to unseen attacks. The results show significant EER reductions (e.g., 36.72% to 7.52% for Qwen2-Audio). However, the experimental setup has limitations: the dataset is relatively small (1,024 samples per environment), and the "unseen attacks" are from the same ASVspoof 2019 test set, which may share underlying characteristics with the training attacks. The comparison with a "Direct" baseline is useful, but the lack of comparison with other state-of-the-art ADD methods (non-ALM based) limits the context of the performance gains. The non-monotonic improvement trajectories are honestly reported.
Reproducibility
The paper provides a GitHub link and details hyperparameters (LoRA rank, learning rate, batch size, etc.). The construction of the evolving environments using the Norton-Bass process is described, but the specific parameters for the activity process are not fully detailed in the text provided. The prompts for cue generation and classification are mentioned as fixed but not provided in the excerpt. Overall, reproducibility is moderate to good, assuming the code repository is complete.
Limitations
1. The dataset size is small, which may limit the robustness of the conclusions. 2. The "unseen" attacks are from the same dataset family (ASVspoof 2019), so true generalization to completely different spoofing techniques (e.g., different TTS engines or vocoders not in the dataset) is not tested. 3. The method relies on the ALM's ability to generate meaningful forensic cues; if the base model is poor at this, the self-evolution loop may be ineffective. 4. The computational cost of iterative LoRA updates and self-generation is not discussed.
Broader impact
The work contributes to the field of AI safety and audio forensics by addressing the challenge of evolving threats. The self-evolving paradigm could be applied to other domains where data distributions shift over time. The use of ALMs for forensic reasoning is a promising direction that could lead to more interpretable and adaptive detection systems. The paper introduces SE-ADD, a self-evolving framework for audio deepfake detection that leverages mistake-driven supervision and self-generated forensic cues to adapt ALMs to evolving spoofing environments, demonstrating significant improvements in generalization to unseen attacks.
Author of this paper? Send a correction.