Breaking the Quality--Intelligibility Trade-off in Streaming Target Speaker Extraction via Deep-Feature-Anchored Preference Optimization
Abstract
Generative streaming models for Target Speaker Extraction (TSE) commonly exhibit a quality--intelligibility trade-off, wherein naive optimization for perceptual audio quality tends to degrade speech intelligibility, and conversely. We reveal that this trade-off arises not from the constraints of streaming architectures, but from an inappropriate choice of optimization anchor. Directly optimizing against audio quality metrics induces catastrophic reward hacking, where content critical to pronunciation and intelligibility is systematically erased to maximize a proxy score. To break this bottleneck, we propose two complementary improvements: an enlarged Conformer convolution kernel for richer local spectro-temporal modeling, and WavLM-anchored Direct Preference Optimization (DPO) fine-tuning strategy. DPO preference pairs are ranked by WavLM cosine similarity, a deep acoustic feature encoding both phonetic structure and speaker identity, providing an optimization anchor that resists hacking. Under a 560 ms streaming chunk size, the proposed method achieves a 10.9% relative intelligibility improvement (word error rate: 0.138 to 0.123), with marginal simultaneous gains in audio quality and speaker similarity.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper makes a significant contribution to streaming TSE by identifying and resolving the reward hacking problem in DPO through deep-feature anchoring, achieving a Pareto improvement in quality and intelligibility.
Methodology
The paper proposes a novel approach to the quality-intelligibility trade-off in streaming Target Speaker Extraction (TSE). The core methodological contribution is the identification and mitigation of "reward hacking" in Direct Preference Optimization (DPO) when applied to generative speech models. The authors correctly diagnose that optimizing for perceptual metrics (DNSMOS) leads to intelligibility collapse, while optimizing for Word Error Rate (WER) fails to improve quality. Their solution involves two steps: architectural (enlarging Conformer kernel size to $k=15$ for better local context) and algorithmic (using WavLM cosine similarity as the preference anchor for DPO). The separation of the semantic pathway (optimized via DPO) and acoustic pathway (frozen) is a sound engineering decision to prevent semantic drift during acoustic refinement. The use of WavLM, which encodes both phonetic and speaker information, as a proxy for "good" speech is a clever heuristic that effectively balances the competing objectives.
Results & evaluation
The experimental setup is rigorous, utilizing the Libri2Mix dataset and standard metrics (DNSMOS, WER, Speaker Similarity). The ablation studies are particularly strong, clearly demonstrating the failure modes of $DPO_{DNSMOS}$ (intelligibility collapse) and $DPO_{WER}$ (no gain, quality drop), thereby validating the necessity of the $DPO_{WavLM}$ approach. The results show a statistically significant improvement in WER (10.9% relative) with simultaneous gains in quality and speaker similarity. The inclusion of Inference Success Rate (ISR) is a valuable addition for streaming contexts. The evaluation covers multiple KL penalty coefficients and checkpoints, providing a robust analysis of hyperparameter sensitivity.
Reproducibility
The paper provides sufficient implementation details, including model architecture (Conformer kernel size, SELM/ARLM layers), training hyperparameters (learning rate, optimizer), and dataset specifications. The use of standard libraries (FunCodec, Whisper, WavLM) aids reproducibility. However, the exact code for the "Historical Context Refinement" mechanism and the specific stochastic sampling procedure for generating preference pairs could be more detailed. The claim of "catastrophic reward hacking" is well-supported by the provided analysis (spectrograms and metric trends), which enhances trust in the reproducibility of the findings.
Limitations
The primary limitation is the reliance on WavLM as a proxy for human preference. While effective, WavLM is a self-supervised model trained on large-scale speech data, and its alignment with human perceptual quality is not perfect. The method is evaluated only on Libri2Mix, a relatively clean, two-speaker mixture dataset. Real-world scenarios with heavy noise, reverberation, and more than two speakers are not addressed. Furthermore, the streaming latency is 560ms, which is acceptable for many applications but not for strict real-time (<200ms) interaction, as noted by the authors. The computational cost of generating 16 candidates for DPO training is also non-trivial.
Broader impact
This work has significant implications for the deployment of generative AI in real-time communication systems, such as teleconferencing and voice assistants. By breaking the quality-intelligibility trade-off, it enables systems that are both natural-sounding and easy to understand. The insight regarding reward hacking in generative speech models is broadly applicable to other text-to-speech and audio generation tasks. It highlights the danger of using naive proxy metrics for optimization and advocates for more semantically aware anchors. The paper makes a significant contribution to streaming TSE by identifying and resolving the reward hacking problem in DPO through deep-feature anchoring, achieving a Pareto improvement in quality and intelligibility.
Author of this paper? Send a correction.