Attention-based Mixture of Experts for Robust Speech Deepfake Detection
Abstract
AI-generated speech is becoming increasingly used in everyday life, powering virtual assistants, accessibility tools, and other applications. However, it is also being exploited for malicious purposes such as impersonation, misinformation, and biometric spoofing. As speech deepfakes become nearly indistinguishable from real human speech, the need for robust detection methods and effective countermeasures has become critically urgent. In this paper, we present the ISPL's submission to the SAFE challenge at IH&MMSec 2025, where our system ranked first across all tasks. Our solution introduces a novel approach to audio deepfake detection based on a Mixture of Experts architecture. The proposed system leverages multiple state-of-the-art detectors, combining their outputs through an attention-based gating network that dynamically weights each expert based on the input speech signal. In this design, each expert develops a specialized understanding of the shared training data by learning to capture different complementary aspects of the same input through inductive biases. Experimental results indicate that our method outperforms existing approaches across multiple datasets. We further evaluate and analyze the performance of our system in the SAFE challenge.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of an attention-based Mixture of Experts architecture for robust speech deepfake detection, which combines the strengths of multiple detectors to improve performance. This innovative approach, along with strong experimental results, positions the work as a valuable addition to the ongoing efforts in combating audio deepfakes, though it requires further details for reproducibility and practical application.
Methodology
The paper introduces a Mixture of Experts (MoE) architecture enhanced by an attention-based gating mechanism, which is a sophisticated approach to audio deepfake detection. The use of multiple state-of-the-art detectors allows the model to leverage complementary strengths, effectively addressing the challenge of distinguishing between real and synthetic speech. The attention mechanism dynamically adjusts the contribution of each expert based on the input, which is a notable innovation that enhances the model's adaptability and robustness. However, the paper could benefit from a more detailed description of the inductive biases employed and how they are integrated into the learning process.
Results & evaluation
The experimental results demonstrate that the proposed method outperforms existing approaches across multiple datasets, which is a strong indicator of its effectiveness. The authors participated in the SAFE challenge, achieving first place across all tasks, which adds credibility to their claims. However, the paper lacks detailed information about the datasets used, including their sizes, diversity, and how they were split for training and testing. This information is crucial for assessing the generalizability of the results.
Reproducibility
The paper does not provide sufficient implementation details or access to code repositories, which raises concerns about reproducibility. While the methodology is described, the absence of a clear path for other researchers to replicate the experiments limits the impact of the findings. Providing a GitHub repository or similar would greatly enhance the paper's contribution to the field.
Limitations
One limitation is the potential overfitting to the datasets used, especially if they are not sufficiently diverse. Additionally, the reliance on multiple experts may increase computational complexity, which could hinder real-time applications. The paper does not address how the model performs under adversarial conditions or with varying qualities of input audio, which is critical for practical deployment.
Broader impact
The implications of this research are significant, particularly in the context of increasing concerns about misinformation and biometric spoofing. Effective detection methods for speech deepfakes can enhance security in various applications, including virtual assistants and online communications. However, the potential for misuse of such technologies also warrants careful consideration of ethical implications and the need for responsible deployment. The main contribution of this paper is the introduction of an attention-based Mixture of Experts architecture for robust speech deepfake detection, which combines the strengths of multiple detectors to improve performance. This innovative approach, along with strong experimental results, positions the work as a valuable addition to the ongoing efforts in combating audio deepfakes, though it requires further details for reproducibility and practical application.
Author of this paper? Send a correction.