YingMusic-SVC: Real-World Robust Zero-Shot Singing Voice Conversion with Flow-GRPO and Singing-Specific Inductive Biases

Gongyu Chen, Xiaoyu Zhang, Zhenqiang Weng ...Full author list in the original paper.

East China University of Science and Technology +2 · arXiv

Affiliations: East China University of Science and Technology, Tsinghua University, University College London (UCL)

Abstract

Singing voice conversion (SVC) aims to render the target singer's timbre while preserving melody and lyrics. However, existing zero-shot SVC systems remain fragile in real songs due to harmony interference, F0 errors, and the lack of inductive biases for singing. We propose YingMusic-SVC, a robust zero-shot framework that unifies continuous pre-training, robust supervised fine-tuning, and Flow-GRPO reinforcement learning. Our model introduces a singing-trained RVC timbre shifter for timbre-content disentanglement, an F0-aware timbre adaptor for dynamic vocal expression, and an energy-balanced rectified flow matching loss to enhance high-frequency fidelity. Experiments on a graded multi-track benchmark show that YingMusic-SVC achieves consistent improvements over strong open-source baselines in timbre similarity, intelligibility, and perceptual naturalness, especially under accompanied and harmony-contaminated conditions, demonstrating its effectiveness for real-world SVC deployment.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

The main contribution of this paper is the development of YingMusic-SVC, a robust zero-shot singing voice conversion framework that effectively integrates singing-specific inductive biases and advanced training methodologies to achieve high-quality voice conversion in real-world scenarios. This work represents a significant step forward in the field of audio processing, addressing critical challenges and paving the way for future innovations in singing voice conversion technology.

Methodology

The proposed YingMusic-SVC framework presents a comprehensive approach to singing voice conversion (SVC) by integrating continuous pre-training, robust supervised fine-tuning, and reinforcement learning. The introduction of singing-specific modules, such as the RVC timbre shifter and F0-aware timbre adaptor, demonstrates a thoughtful adaptation of existing methodologies to better suit the unique characteristics of singing voices. The use of energy-balanced flow matching loss further enhances the model's ability to capture high-frequency details, which is critical for realistic singing voice conversion. The multi-stage training approach is well-structured and effectively addresses the challenges posed by real-world audio scenarios.

Results & evaluation

The experiments conducted on a graded multi-track benchmark are extensive and provide a solid basis for evaluating the proposed system's performance. The results indicate significant improvements over existing baselines in various metrics, including timbre similarity, intelligibility, and perceptual naturalness, particularly in challenging conditions with background music. The use of both objective and subjective evaluation metrics strengthens the findings, showcasing the model's robustness and effectiveness in real-world applications.

Reproducibility

The paper provides sufficient implementation details, including the architecture, training data, and evaluation metrics, which facilitate reproducibility. The open-source availability of the code and models further enhances the potential for other researchers to replicate and build upon this work.

Limitations

While the proposed method shows promising results, it may still struggle with extreme vocal ranges or highly complex musical arrangements that deviate significantly from the training data. Additionally, the reliance on a well-structured training dataset may limit the model's generalizability to less common singing styles or languages not represented in the training corpus.

Broader impact

The advancements presented in YingMusic-SVC have significant implications for various applications, including music production, virtual singers, and content creation in social media. By improving the quality and robustness of singing voice conversion, this work could enhance user experiences in entertainment and creative industries, potentially leading to new forms of artistic expression and collaboration. The main contribution of this paper is the development of YingMusic-SVC, a robust zero-shot singing voice conversion framework that effectively integrates singing-specific inductive biases and advanced training methodologies to achieve high-quality voice conversion in real-world scenarios. This work represents a significant step forward in the field of audio processing, addressing critical challenges and paving the way for future innovations in singing voice conversion technology.

Author of this paper? Send a correction.