SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings
Abstract
We introduce the REAL-TSE Challenge, an IEEE SLT 2026 satellite challenge on target speaker extraction~(TSE) from real conversational recordings. Given a multi-speaker mixture and one or more enrollment utterances from a target speaker, participating systems must recover only the target speech. Unlike simulated read-speech benchmarks, REAL-TSE evaluates Mandarin and English recordings that contain natural overlap, reverberation, noise, channel mismatch, and conversational dynamics. The challenge defines two complementary tracks: an Online track for low-latency streaming extraction and an Offline track for full-context processing. Systems are evaluated with Token Error Rate (TER), Speaker Similarity (SpkSim), DNSMOS, and target-speaker activity F1. This overview paper describes the task definition, datasets, baselines, evaluation protocol, submitted systems, condition-wise findings, and lessons for future real-world TSE benchmarks.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The REAL-TSE Challenge establishes a critical new benchmark for real-world Target Speaker Extraction, revealing significant gaps between simulated and real-world performance and highlighting the dangers of metric over-optimization in neural audio evaluation.
Methodology
This paper introduces the REAL-TSE Challenge, a benchmarking framework rather than a novel algorithmic architecture. The methodological contribution lies in the rigorous definition of a real-world Target Speaker Extraction (TSE) task. It distinguishes itself from prior simulated benchmarks (like LibriMix) by utilizing real conversational recordings with natural overlap, reverberation, noise, and channel mismatch. The challenge is structured into two tracks: Online (low-latency, <100ms) and Offline (full-context). The evaluation methodology is multi-dimensional, employing Token Error Rate (TER), Speaker Similarity (SpkSim), DNSMOS-P808, and Target-Speaker Activity F1. A critical methodological insight presented is the detection of metric over-optimization, specifically regarding DNSMOS-OVRL, leading to the adoption of DNSMOS-P808 for ranking. The paper details the data simulation strategies used by participants (pseudo-labeling, real-data adaptation) and analyzes the impact of channel mismatch and scenario types on performance.
Results & evaluation
The experimental section provides a comprehensive analysis of 24 systems from 18 teams. It includes baseline results (BSRNN variants) and aggregate performance metrics. The paper conducts a detailed condition-wise analysis on the EVAL-2 set, examining performance across different recording devices (H1, H2, Phone), target-speaker activity ratios, and recording scenarios (meeting, cafe, home, vehicle). Key findings include the significant impact of mixture recording device quality, the lack of consistent correlation between nominal scenario labels and difficulty, and the trade-offs between different metrics (e.g., systems optimizing for SpkSim may suffer in TER). The analysis of DNSMOS reliability is a significant experimental contribution, demonstrating how neural metrics can be gamed without perceptual improvement.
Reproducibility
The challenge promotes reproducibility and transparency. The authors release the evaluation toolkit, baseline code (WeSep-based), and the dataset splits (Development and Evaluation). The training policy is open but restricted regarding specific test-set leakage (forbidden to use dev/test splits of source corpora for training). The paper provides detailed descriptions of the latency verification process (perturbation-based) and metric computation. However, as a challenge overview, it does not provide full training code for the winning systems, which is standard for such venues, but it does release the infrastructure for future replication.
Limitations
The primary limitation is the inherent difficulty of real-world evaluation: the lack of ground-truth clean targets for most real recordings makes absolute performance measurement challenging, relying instead on proxies like ASR and speaker similarity. The paper acknowledges that metric-aware over-optimization is a significant issue, suggesting that current automatic metrics may not fully align with human perception or true extraction quality. Additionally, the "unseen" nature of EVAL-2 means that generalization to entirely new, unrecorded environments remains an open question, though the challenge design attempts to mitigate this by using diverse, newly recorded data.
Broader impact
This work has a substantial impact on the speech processing community by shifting the focus from simulated to real-world TSE evaluation. It highlights the gap between lab-performance and real-world deployment, emphasizing the need for robustness to channel mismatch, noise, and conversational dynamics. The findings on metric reliability (DNSMOS) have broader implications for the entire field of audio quality assessment and neural metric usage. By providing a standardized benchmark and revealing the limitations of current approaches, it guides future research towards more practical, multi-objective optimization and better data simulation techniques. The REAL-TSE Challenge establishes a critical new benchmark for real-world Target Speaker Extraction, revealing significant gaps between simulated and real-world performance and highlighting the dangers of metric over-optimization in neural audio evaluation.
Author of this paper? Send a correction.