Rethinking Training Targets, Architectures and Data Quality for Universal Speech Enhancement
Abstract
Universal Speech Enhancement (USE) aims to restore speech quality under diverse degradation conditions while preserving signal fidelity. Despite recent progress, key challenges in training target selection, the distortion--perception tradeoff, and data curation remain unresolved. In this work, we systematically address these three overlooked problems. First, we revisit the conventional practice of using early-reflected speech as the dereverberation target and show that it can degrade perceptual quality and downstream ASR performance. We instead demonstrate that time-shifted anechoic clean speech provides a superior learning target. Second, guided by the distortion--perception tradeoff theory, we propose a simple two-stage framework that achieves minimal distortion under a given level of perceptual quality. Third, we analyze the trade-off between training data scale and quality for USE, revealing that training on large uncurated corpora imposes a performance ceiling, as models struggle to remove subtle artifacts. Our method achieves state-of-the-art performance on the URGENT 2025 non-blind test set and exhibits strong language-agnostic generalization, making it effective for improving TTS training data. Code and models will be released upon acceptance.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of a novel approach to Universal Speech Enhancement that significantly improves speech quality and ASR performance by rethinking training targets and leveraging a two-stage model framework. This work addresses critical gaps in the field and provides a solid foundation for future research and applications in speech processing.
Methodology
The paper presents a systematic approach to Universal Speech Enhancement (USE) by addressing three critical challenges: training target selection, the distortion-perception tradeoff, and data quality. The authors propose using time-shifted anechoic clean speech as a learning target, which is shown to outperform conventional early-reflected speech. They also introduce a two-stage framework that combines regression and generative models to balance fidelity and perceptual quality effectively. This methodology is well-grounded in theoretical principles and is supported by empirical evidence.
Results & evaluation
The experiments are comprehensive, utilizing the URGENT 2025 Challenge dataset, which includes diverse speech distortions and languages. The authors provide detailed results that demonstrate significant improvements in both perceptual quality and automatic speech recognition (ASR) performance. The evaluation metrics are robust, covering both intrusive and non-intrusive measures, which strengthens the validity of their findings.
Reproducibility
The authors commit to releasing their code and models upon acceptance, which is a positive step towards reproducibility. However, the paper could benefit from more detailed descriptions of the experimental setup, such as hyperparameters and specific training procedures, to facilitate replication.
Limitations
One notable limitation is the reliance on the URGENT 2025 Challenge dataset, which may not fully represent real-world conditions. Additionally, while the proposed method shows improvements, the paper does not extensively discuss scenarios where the model may fail or the potential for overfitting to the training data.
Broader impact
The advancements in speech enhancement have significant implications for various applications, including telecommunications, assistive technologies, and improving the quality of training data for text-to-speech systems. The language-agnostic nature of the proposed method could also benefit low-resource languages, enhancing accessibility and communication. The main contribution of this paper is the introduction of a novel approach to Universal Speech Enhancement that significantly improves speech quality and ASR performance by rethinking training targets and leveraging a two-stage model framework. This work addresses critical gaps in the field and provides a solid foundation for future research and applications in speech processing.
Author of this paper? Send a correction.