VorTEX: Various overlap ratio for Target speech EXtraction
Abstract
Target speech extraction (TSE) aims to recover a target speaker's voice from a mixture. While recent text-prompted approaches have shown promise, most approaches assume fully overlapped mixtures, limiting insight into behavior across realistic overlap ratios. We introduce VorTEX (Various overlap ratio for Target speech EXtraction), a text-prompted TSE architecture with a Decoupled Adaptive Multi-branch (DAM) Fusion block that separates primary extraction from auxiliary regularization pathways. To enable controlled analysis, we construct PORTE, a two-speaker dataset spanning overlap ratios from 0% to 100%. We further propose Suppression Ratio on Energy (SuRE), a diagnostic metric that detects suppression behavior not captured by conventional measures. Experiments show that existing models exhibit suppression or residual interference under overlap, whereas VorTEX achieves the highest separation fidelity across 20-100% overlap (e.g., 5.50 dB at 20% and 2.04 dB at 100%) while maintaining zero SuRE, indicating robust extraction without suppression-driven artifacts.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this work is the introduction of VorTEX, a robust text-prompted TSE model that effectively addresses the challenges of varying overlap ratios in speech mixtures, alongside the creation of the PORTE dataset for evaluating TSE performance. This research significantly advances the field of audio processing by providing a novel architecture and evaluation framework that can lead to improved speech extraction in practical applications.
Methodology
The paper introduces VorTEX, a novel architecture for target speech extraction (TSE) that utilizes a Decoupled Adaptive Multi-branch (DAM) Fusion block to separate extraction and regularization pathways. This approach is innovative as it addresses the limitations of existing models that primarily focus on fully overlapped mixtures, thus enhancing the robustness of TSE across various overlap ratios. The proposed methodology is well-structured, with a clear explanation of the DAM architecture and its components, including Multi-Scale Fusion, Adaptive Fusion, and Dual Projection Fusion. The introduction of the PORTE dataset is a significant contribution, providing a controlled environment for evaluating TSE models under realistic conditions.
Results & evaluation
The experiments conducted are thorough, comparing VorTEX against established models in the field, such as AudioSep and DGMO, as well as other text-prompted TSE models like StyleTSE and LLM-TSE. The use of multiple evaluation metrics, including SISDR, PESQ, and the newly proposed SuRE metric, allows for a comprehensive assessment of model performance. The results demonstrate VorTEX's superior extraction fidelity and robustness, particularly in high-overlap scenarios, validating the effectiveness of the proposed architecture.
Reproducibility
The paper provides sufficient details regarding the architecture, training configuration, and evaluation metrics, which would allow for reproducibility. However, the lack of a public repository or demo URL limits the ease of access for other researchers to replicate the results.
Limitations
While the study presents significant advancements, it relies on a synthetic dataset (PORTE), which may not fully capture the complexities of real-world conversational audio. Additionally, the prompts used for TSE are limited to observable attributes, suggesting that future work could explore more complex prompt structures. The paper also acknowledges the need for further research to develop metrics that comprehensively assess extraction fidelity, perceptual quality, and speaker preservation.
Broader impact
The findings of this research have the potential to improve applications in speech recognition, assistive technologies, and audio processing systems where clear target speech extraction is crucial. By addressing the challenges of overlapping speech, VorTEX could enhance user experiences in various real-world scenarios, such as in crowded environments or during multi-speaker conversations. The main contribution of this work is the introduction of VorTEX, a robust text-prompted TSE model that effectively addresses the challenges of varying overlap ratios in speech mixtures, alongside the creation of the PORTE dataset for evaluating TSE performance. This research significantly advances the field of audio processing by providing a novel architecture and evaluation framework that can lead to improved speech extraction in practical applications.
Author of this paper? Send a correction.