TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation
Abstract
Source separation is a fundamental task in speech, music, and audio processing, and it also provides cleaner and larger data for training generative models. However, improving separation performance in practice often depends on increasingly large networks, inflating training and deployment costs. Motivated by recent advances in inference-time scaling for generative modeling, we propose Training-Time and Inference-Time Scalable Discriminative Source Separation (TISDiSS), a unified framework that integrates early-split multi-loss supervision, shared-parameter design, and dynamic inference repetitions. TISDiSS enables flexible speed-performance trade-offs by adjusting inference depth without retraining additional models. We further provide systematic analyses of architectural and training choices and show that training with more inference repetitions improves shallow-inference performance, benefiting low-latency applications. Experiments on standard speech separation benchmarks demonstrate state-of-the-art performance with a reduced parameter count, establishing TISDiSS as a scalable and practical framework for adaptive source separation. Code is available at https://github.com/WingSingFung/TISDiSS.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of the TISDiSS framework, which effectively balances performance and computational efficiency in discriminative source separation tasks. This work presents a significant advancement in the field, particularly for applications requiring low-latency processing, while also providing a solid foundation for future research in scalable audio processing methodologies.
Methodology
The proposed TISDiSS framework integrates several innovative components, including early-split multi-loss supervision and shared-parameter design, which are well-justified in the context of improving source separation tasks. The dynamic inference repetitions allow for a flexible trade-off between speed and performance, which is particularly relevant for real-time applications. However, while the methodology is robust, the paper could benefit from a clearer explanation of how these components interact and their specific contributions to the overall performance improvement.
Results & evaluation
The experiments are conducted on standard speech separation benchmarks, showcasing state-of-the-art performance with a reduced parameter count. The results are compelling and demonstrate the effectiveness of the TISDiSS framework. However, the paper lacks a comprehensive comparison with a broader range of existing methods, which could further validate the claimed advantages.
Reproducibility
The authors provide a GitHub repository for code access, which is a positive aspect for reproducibility. However, the paper would benefit from more detailed documentation regarding the experimental setup, hyperparameters, and specific configurations used in the experiments to facilitate easier reproduction by other researchers.
Limitations
One limitation is the potential overfitting to the specific benchmarks used, as the paper does not explore the generalizability of the TISDiSS framework across diverse datasets or tasks beyond speech separation. Additionally, the reliance on dynamic inference repetitions may introduce complexity in deployment, which could be a barrier for practical applications.
Broader impact
The TISDiSS framework has significant implications for real-time audio processing applications, such as virtual assistants and music production tools, where efficient source separation is crucial. By enabling scalable performance adjustments, it opens avenues for further research into adaptive models that can cater to varying computational resources. The main contribution of this paper is the introduction of the TISDiSS framework, which effectively balances performance and computational efficiency in discriminative source separation tasks. This work presents a significant advancement in the field, particularly for applications requiring low-latency processing, while also providing a solid foundation for future research in scalable audio processing methodologies.
Author of this paper? Send a correction.