Input-Adaptive Spectral Feature Compression by Sequence Modeling for Source Separation
Abstract
Time-frequency domain dual-path models have demonstrated strong performance and are widely used in source separation. Because their computational cost grows with the number of frequency bins, these models often use the band-split (BS) module in high-sampling-rate tasks such as music source separation (MSS) and cinematic audio source separation (CASS). The BS encoder compresses frequency information by encoding features for each predefined subband. It achieves effective compression by introducing an inductive bias that places greater emphasis on low-frequency parts. Despite its success, the BS module has two inherent limitations: (i) it is not input-adaptive, preventing the use of input-dependent information, and (ii) the parameter count is large, since each subband requires a dedicated module. To address these issues, we propose Spectral Feature Compression (SFC). SFC compresses the input using a single sequence modeling module, making it both input-adaptive and parameter-efficient. We investigate two variants of SFC, one based on cross-attention and the other on Mamba, and introduce inductive biases inspired by the BS module to make them suitable for frequency information compression. Experiments on MSS and CASS tasks demonstrate that the SFC module consistently outperforms the BS module across different separator sizes and compression ratios. We also provide an analysis showing that SFC adaptively captures frequency patterns from the input.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of the Spectral Feature Compression module, which provides a novel, input-adaptive, and parameter-efficient approach to spectral feature compression for source separation tasks. This work represents a meaningful advancement in the field of audio processing, addressing key limitations of existing methods and demonstrating strong empirical results.
Methodology
The paper introduces a novel approach to spectral feature compression through the Spectral Feature Compression (SFC) module, which utilizes sequence modeling techniques to create an input-adaptive and parameter-efficient method for source separation. The methodology is well-structured, addressing the limitations of the traditional band-split (BS) module by incorporating inductive biases and demonstrating the effectiveness of two variants based on cross-attention and Mamba. The approach is innovative in its attempt to adaptively capture frequency patterns, which is a significant advancement over previous methods.
Results & evaluation
The experiments are comprehensive, evaluating the proposed SFC module against the BS module across various tasks, including music source separation (MSS) and cinematic audio source separation (CASS). The results consistently show that SFC outperforms BS across different separator sizes and compression ratios, indicating a robust experimental design. However, details on the datasets used and the specific metrics for evaluation could be elaborated further to enhance clarity.
Reproducibility
The paper lacks specific implementation details that would facilitate reproducibility, such as code availability or detailed descriptions of the experimental setup. While the methodology is sound, the absence of a project URL or demo could hinder other researchers from replicating the results.
Limitations
One limitation is the reliance on inductive biases inspired by the BS module, which may not generalize well to all types of audio signals. Additionally, while the SFC module shows promise, its performance in real-world scenarios beyond the tested datasets remains unverified.
Broader impact
The proposed method has significant implications for audio processing applications, particularly in enhancing the quality of source separation in music and cinematic audio. The input-adaptive nature of the SFC module could lead to more efficient and effective audio processing systems, potentially influencing both academic research and industry practices. The main contribution of this paper is the introduction of the Spectral Feature Compression module, which provides a novel, input-adaptive, and parameter-efficient approach to spectral feature compression for source separation tasks. This work represents a meaningful advancement in the field of audio processing, addressing key limitations of existing methods and demonstrating strong empirical results.
Author of this paper? Send a correction.