Probing Token Spaces under Generator Shift in AI-Generated Music Detection
Abstract
AI-generated music detectors can appear robust on standard benchmark splits, yet their deployments require transfer to generator sources absent during training. We study this problem with source-restricted evaluation on \textsc{MoM-open}, an open reconstruction of MoM-CLAM that replaces the non-redistributable real corpus with FMA and MTG-Jamendo while preserving the fake-generator protocol. To isolate the role of representation, we introduce \textsc{CoMoE}, a compact fixed classifier for comparing heterogeneous audio token spaces while keeping the downstream architecture and training recipe unchanged. Experiments show that standard and real-source-restricted splits are nearly saturated, whereas fake-source restriction exposes large differences between token spaces: X-Codec tokens are strongest when training on Udio alone, while MERT-derived tokens are stronger when training on Suno-v3.5 alone. These results suggest that codec-style discrete token spaces should be treated as a primary experimental axis under generator shift in AI-generated music detection. Our code and data are available at https://github.com/MAAP-LAB/CoMoE.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a controlled study of cross-generator AI-generated music detection, introducing a novel approach to evaluating the impact of different audio token spaces on detection performance. This work is significant as it not only advances the understanding of AI-generated music detection but also provides a framework for future research in the field.
Methodology
The paper introduces CoMoE, a novel compact classifier designed to evaluate the impact of different audio tokenizers on AI-generated music detection. The methodology is robust, leveraging a controlled experimental setup where only the input token space varies while keeping the downstream architecture constant. This design allows for a clear isolation of the effects of different tokenizers on detection performance, which is a significant advancement in the field. The use of codec-style discrete audio tokens as a primary experimental axis is innovative, as it shifts the focus from traditional continuous representations to a more forensic approach that may yield better insights into the characteristics of AI-generated music.
Results & evaluation
The experiments are well-structured, utilizing the MoM-open dataset, which is a thoughtful reconstruction of existing benchmarks. The authors provide a comprehensive analysis of the performance of various tokenizers under different evaluation conditions, revealing critical insights into the robustness of AI-generated music detectors. The results indicate that the choice of tokenizer significantly affects detection performance, particularly under generator shift scenarios. The evaluation metrics used, including AUC and held-out-fake detection rates, are appropriate for the task and provide a clear picture of the model's performance across different conditions.
Reproducibility
The paper includes sufficient details regarding the architecture of CoMoE, the training protocols, and the datasets used, which enhances reproducibility. The authors have made their code and data publicly available, which is a positive aspect for the community and encourages further exploration and validation of their findings.
Limitations
The study acknowledges limitations, such as the reliance on an open reconstruction of the MoM-CLAM dataset and potential biases introduced by the X-Codec mini tokenizer. Additionally, the authors suggest that future work should explore more generator sources and control training-pool sizes, indicating areas where the current study could be expanded or improved.
Broader impact
The findings of this research have significant implications for the field of AI-generated music detection, particularly in improving the robustness of detection systems against unseen generator sources. By emphasizing the importance of tokenizer choice, this work could influence future research directions and methodologies in music AI, potentially leading to more reliable detection systems that can be deployed in real-world applications. The paper presents a controlled study of cross-generator AI-generated music detection, introducing a novel approach to evaluating the impact of different audio token spaces on detection performance. This work is significant as it not only advances the understanding of AI-generated music detection but also provides a framework for future research in the field.
Author of this paper? Send a correction.