Beyond Artifacts: Towards Generalizable Synthetic Song Detection via Music-Intrinsic Features
Abstract
The rapid advancement of AI music generators highlights the urgent need for reliable Synthetic Song Detection (SSD). Existing SSD methods often rely on low-level artifacts or fixed feature assumptions, struggling to capture generator-agnostic cues. To address this, we propose Sofia (Synthetic-song detection framework via music features), a flexible framework that models music-intrinsic attributes via feature-specific experts and an adaptive Mixture-of-Experts (MoE) module. By configuring Sofia with representative Vocal, Audio-effect, Global structure features, and their combinations, we present their individual and complementary contributions. To comprehensively evaluate our framework, we further construct MUSIC8K, a challenging benchmark featuring lastest emerging generators and realistic audio perturbations. Experiments show that Sofia learns generator-agnostic representations from music-intrinsic features, improving the F1 score by 18.5 points over the strongest baseline on MUSIC8K-O while maintaining strong robustness.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents Sofia, a flexible framework for synthetic song detection that leverages music-intrinsic features through a Mixture-of-Experts approach. This innovative methodology addresses existing limitations in SSD methods, providing a significant contribution to the field of audio analysis and detection.
Methodology
The paper introduces Sofia, a novel framework for Synthetic Song Detection (SSD) that utilizes a Mixture-of-Experts (MoE) approach to model music-intrinsic features. This method allows for flexible feature incorporation, enabling the framework to adapt to various music generators and improve generalization. The use of feature-specific experts to capture distinct musical attributes is a significant methodological advancement, as it addresses the limitations of existing SSD methods that rely on low-level artifacts or fixed feature assumptions. The framework's design supports systematic analysis of individual and complementary contributions of different music features, which is a notable strength.
Results & evaluation
The authors construct the MUSIC8K dataset, which serves as a benchmark for evaluating the performance of SSD methods against the latest music generators and realistic audio perturbations. The experiments demonstrate that Sofia achieves a substantial improvement in F1 score (18.5 points) over the strongest baseline, showcasing its effectiveness in learning generator-agnostic representations. The comprehensive evaluation across various configurations and the inclusion of robustness testing against audio perturbations further validate the framework's performance and adaptability.
Reproducibility
The paper provides detailed implementation information, including training configurations, audio preprocessing, and encoder settings, which enhances reproducibility. The availability of the code and dataset on GitHub and Hugging Face facilitates further experimentation and validation by other researchers in the field.
Limitations
While the framework shows promise, it primarily explores one instantiation (Sofia-VAG) based on selected music features. As music generation technology evolves, the reliance on a fixed set of features may limit the framework's applicability to future generators. Additionally, the paper does not address the potential computational overhead associated with the MoE architecture, which could impact real-time applications.
Broader impact
The development of a robust SSD framework has significant implications for the music industry, particularly in combating the proliferation of synthetic music and ensuring authenticity in audio content. The ability to detect synthetic songs effectively can enhance content moderation on platforms that host user-generated music, thereby preserving the integrity of artistic expression. Furthermore, the framework's adaptability to new music generation technologies positions it as a valuable tool for future research in audio analysis and detection. The paper presents Sofia, a flexible framework for synthetic song detection that leverages music-intrinsic features through a Mixture-of-Experts approach. This innovative methodology addresses existing limitations in SSD methods, providing a significant contribution to the field of audio analysis and detection.
Author of this paper? Send a correction.