Controllable Singing Style Conversion with Boundary-Aware Information Bottleneck
Abstract
This paper presents the submission of the S4 team to the Singing Voice Conversion Challenge 2025 (SVCC2025)-a novel singing style conversion system that advances fine-grained style conversion and control within in-domain settings. To address the critical challenges of style leakage, dynamic rendering, and high-fidelity generation with limited data, we introduce three key innovations: a boundary-aware Whisper bottleneck that pools phoneme-span representations to suppress residual source style while preserving linguistic content; an explicit frame-level technique matrix, enhanced by targeted F0 processing during inference, for stable and distinct dynamic style rendering; and a perceptually motivated high-frequency band completion strategy that leverages an auxiliary standard 48kHz SVC model to augment the high-frequency spectrum, thereby overcoming data scarcity without overfitting. In the official SVCC2025 subjective evaluation, our system achieves the best naturalness performance among all submissions while maintaining competitive results in speaker similarity and technique control, despite using significantly less extra singing data than other top-performing systems. Audio samples are available online.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of a controllable singing style conversion system that effectively mitigates style leakage and enhances dynamic rendering through innovative methodologies. This work significantly advances the state of the art in singing voice conversion, demonstrating high fidelity and naturalness even with limited training data, and sets a strong foundation for future research in this domain.
Methodology
The paper introduces a novel approach to singing style conversion that effectively addresses style leakage and dynamic rendering issues through a boundary-aware semantic bottleneck and an explicit technique matrix. The methodology is well-structured, leveraging phoneme-level pooling to enhance control over the conversion process. The use of auxiliary models for high-frequency band completion is particularly innovative, allowing the authors to achieve high fidelity despite data limitations. The integration of targeted pitch processing during inference further enhances the system's performance, demonstrating a comprehensive understanding of the challenges in singing voice conversion.
Results & evaluation
The experimental setup is robust, with a clear description of the training and evaluation processes. The authors conducted subjective evaluations in the SVCC2025 challenge, achieving the best naturalness score among all submissions, which underscores the effectiveness of their approach. The ablation studies provide valuable insights into the contributions of various components, validating the importance of the boundary-aware pooling and technique matrix in reducing style leakage and improving controllability.
Reproducibility
The paper provides sufficient details regarding the methodology and experimental setup, including the training stages and the use of specific models and datasets. The availability of the source code on GitHub enhances reproducibility, allowing other researchers to replicate the experiments and build upon the findings.
Limitations
While the paper presents significant advancements, it does not address the potential challenges of generalizing the model to out-of-domain singing styles or the limitations of the dataset used for training. Additionally, the reliance on specific phoneme boundaries and technique annotations may limit the model's applicability in more diverse or less structured datasets.
Broader impact
The advancements in controllable singing style conversion have implications for various applications, including music production, voice synthesis for entertainment, and personalized audio experiences. The techniques developed could also be adapted for other audio processing tasks, contributing to the broader field of generative audio systems. The main contribution of this paper is the introduction of a controllable singing style conversion system that effectively mitigates style leakage and enhances dynamic rendering through innovative methodologies. This work significantly advances the state of the art in singing voice conversion, demonstrating high fidelity and naturalness even with limited training data, and sets a strong foundation for future research in this domain.
Author of this paper? Send a correction.