A Data-Centric Approach to Generalizable Speech Deepfake Detection
Abstract
Achieving robust generalization in speech deepfake detection (SDD) remains a primary challenge, as models often fail to detect unseen forgery methods. While research has focused on model-centric and algorithm-centric solutions, the impact of data composition is often underexplored. This paper proposes a data-centric approach, analyzing the SDD data landscape from two practical perspectives: constructing a single dataset and aggregating multiple datasets. To address the first perspective, we conduct a large-scale empirical study to characterize the data scaling laws for SDD, quantifying the impact of source and generator diversity. To address the second, we propose the Diversity-Optimized Sampling Strategy (DOSS), a principled framework for mixing heterogeneous data with two implementations: DOSS-Select (pruning) and DOSS-Weight (re-weighting). Our experiments show that DOSS-Select outperforms the naive aggregation baseline while using only 3% of the total available data. Furthermore, our final model, trained on a 12k-hour curated data pool using the optimal DOSS-Weight strategy, achieves state-of-the-art performance, outperforming large-scale baselines with greater data and model efficiency on both public benchmarks and a new challenge set of various commercial APIs.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a novel data-centric approach to generalizable speech deepfake detection, significantly advancing the field by demonstrating the importance of data composition in model performance. The comprehensive methodology and rigorous experimental evaluation underscore its potential impact on future research and practical applications in combating deepfake technologies.
Methodology
The paper introduces a data-centric approach to speech deepfake detection (SDD) by focusing on data composition rather than solely on model architecture. It proposes the Diversity-Optimized Sampling Strategy (DOSS), which includes two implementations: DOSS-Select for pruning and DOSS-Weight for re-weighting. The empirical study on data scaling laws is well-structured, providing a solid foundation for the proposed methods. The methodology is innovative in its systematic analysis of how source and generator diversity impact model generalization, which is a significant shift from traditional model-centric approaches.
Results & evaluation
The experiments are comprehensive, utilizing a large-scale data pool and multiple datasets to validate the proposed methods. The results demonstrate a clear improvement in generalization performance over naive aggregation methods, with DOSS-Weight achieving state-of-the-art results. The evaluation metrics used (EER and ACC) are appropriate for the task, and the experiments are well-documented, allowing for a clear understanding of the impact of the proposed strategies.
Reproducibility
The paper provides sufficient details on the experimental setup, including data generation, model architecture, and training configuration. However, the lack of URLs for code or data repositories limits reproducibility. While the methodology is clear, the absence of a public implementation may hinder other researchers from validating the results independently.
Limitations
The paper acknowledges its focus on English and Chinese languages, which may limit the generalizability of the findings to other languages. Additionally, the study does not explore the interaction between data composition and model scaling, which could be a valuable area for future research. The reliance on a fixed model architecture may also restrict the applicability of the findings to other architectures.
Broader impact
The findings have significant implications for the development of robust speech deepfake detection systems, especially as synthetic speech technologies continue to evolve. By emphasizing data composition, this work encourages future research to explore data-centric approaches in other domains of machine learning, potentially leading to more generalizable models across various applications. The paper presents a novel data-centric approach to generalizable speech deepfake detection, significantly advancing the field by demonstrating the importance of data composition in model performance. The comprehensive methodology and rigorous experimental evaluation underscore its potential impact on future research and practical applications in combating deepfake technologies.
Author of this paper? Send a correction.