AFSS: Artifact-Focused Self-Synthesis for Mitigating Bias in Audio Deepfake Detection
Abstract
The rapid advancement of generative models has enabled highly realistic audio deepfakes, yet current detectors suffer from a critical bias problem, leading to poor generalization across unseen datasets. This paper proposes Artifact-Focused Self-Synthesis (AFSS), a method designed to mitigate this bias by generating pseudo-fake samples from real audio via two mechanisms: self-conversion and self-reconstruction. The core insight of AFSS lies in enforcing same-speaker constraints, ensuring that real and pseudo-fake samples share identical speaker identity and semantic content. This forces the detector to focus exclusively on generation artifacts rather than irrelevant confounding factors. Furthermore, we introduce a learnable reweighting loss to dynamically emphasize synthetic samples during training. Extensive experiments across 7 datasets demonstrate that AFSS achieves state-of-the-art performance with an average EER of 5.45\%, including a significant reduction to 1.23\% on WaveFake and 2.70\% on In-the-Wild, all while eliminating the dependency on pre-collected fake datasets. Our code is publicly available at https://github.com/NguyenLeHaiSonGit/AFSS.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents AFSS, a novel approach to bias mitigation in audio deepfake detection, significantly advancing the field by focusing on generation artifacts while eliminating confounding factors. The methodology's innovative design and rigorous experimental validation position it as a meaningful contribution to audio machine learning research.
Methodology
The proposed Artifact-Focused Self-Synthesis (AFSS) method is innovative in its approach to mitigating bias in audio deepfake detection. By generating pseudo-fake samples through self-conversion and self-reconstruction mechanisms while enforcing same-speaker constraints, the methodology effectively addresses the critical issue of irrelevant confounding factors that typically hinder detection performance across different datasets. The introduction of a learnable reweighting loss further enhances the model's ability to focus on synthetic samples, which is a significant advancement in training strategies for audio deepfake detection.
Results & evaluation
The experiments conducted across seven diverse datasets provide a robust evaluation of the proposed method. The reported results demonstrate state-of-the-art performance with an average Equal Error Rate (EER) of 5.45%, showcasing significant improvements over existing methods, particularly in challenging real-world scenarios. The comprehensive comparison with multiple baselines and disentanglement methods reinforces the effectiveness of AFSS in achieving superior cross-domain generalization.
Reproducibility
The paper provides detailed implementation details, including the architecture, training procedures, and evaluation metrics, which facilitate reproducibility. The availability of the code on GitHub enhances the potential for other researchers to replicate and build upon the work.
Limitations
While the AFSS method shows promise, it may still be limited by the quality and diversity of the real audio samples used for generating pseudo-fake samples. Additionally, the reliance on specific transformation techniques may not generalize to all types of audio deepfakes, potentially limiting the method's applicability in broader contexts.
Broader impact
The implications of this research are significant, as it addresses a pressing need for reliable audio deepfake detection methods in an era where generative models are becoming increasingly sophisticated. The ability to mitigate bias effectively can enhance the robustness of detection systems, contributing to the prevention of misuse in various applications, including security, media integrity, and misinformation. The paper presents AFSS, a novel approach to bias mitigation in audio deepfake detection, significantly advancing the field by focusing on generation artifacts while eliminating confounding factors. The methodology's innovative design and rigorous experimental validation position it as a meaningful contribution to audio machine learning research.
Author of this paper? Send a correction.