When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems
Abstract
Speech enhancement methods are commonly believed to improve the performance of automatic speech recognition (ASR) in noisy environments. However, the effectiveness of these techniques cannot be taken for granted in the case of modern large-scale ASR models trained on diverse, noisy data. We present a systematic evaluation of MetricGAN-plus-voicebank denoising on four state-of-the-art ASR systems: OpenAI Whisper, NVIDIA Parakeet, Google Gemini Flash 2.0, Parrotlet-a using 500 medical speech recordings under nine noise conditions. ASR performance is measured using semantic WER (semWER), a normalized word error rate (WER) metric accounting for domain-specific normalizations. Our results reveal a counterintuitive finding: speech enhancement preprocessing degrades ASR performance across all noise conditions and models. Original noisy audio achieves lower semWER than enhanced audio in all 40 tested configurations (4 models x 10 conditions), with degradations ranging from 1.1% to 46.6% absolute semWER increase. These findings suggest that modern ASR models possess sufficient internal noise robustness and that traditional speech enhancement may remove acoustic features critical for ASR. For practitioners deploying medical scribe systems in noisy clinical environments, our results indicate that preprocessing audio with noise reduction techniques might not just be computationally wasteful but also be potentially harmful to the transcription accuracy.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a critical examination of the effects of speech enhancement on modern ASR systems in medical contexts, revealing that such preprocessing can degrade performance rather than improve it. The comprehensive methodology and significant findings challenge existing paradigms in the field, highlighting the need for further investigation into the relationship between enhancement techniques and ASR performance.
Methodology
The methodology is robust, employing a systematic evaluation of the MetricGAN-plus-voicebank denoising technique across four state-of-the-art ASR systems. The authors clearly define their research question, null hypothesis, and the noise conditions under which the experiments are conducted. The use of a well-defined dataset of medical recordings and the application of semantic WER (semWER) as a performance metric tailored for the medical domain enhances the study's relevance and rigor. However, the study is limited to a single enhancement method, which may not capture the full spectrum of speech enhancement techniques.
Results & evaluation
The experiments are comprehensive, involving 500 medical recordings under nine noise conditions, leading to a total of 40 configurations tested. The results consistently show that speech enhancement degrades ASR performance, which is a significant finding. The detailed analysis of results across different noise types and ASR models provides valuable insights into the interaction between noise and enhancement techniques. However, the reliance on synthetic noise may limit the generalizability of the findings to real-world scenarios.
Reproducibility
The authors have made their evaluation code, dataset, and detailed results publicly available, which greatly enhances reproducibility. The clear description of the methodology and the resources provided allow other researchers to replicate the study effectively.
Limitations
The study's limitations include the focus on a single enhancement method, the use of synthetic noise rather than real-world recordings, and a relatively small dataset size of 500 recordings. Additionally, the findings may not generalize to other ASR systems or domains outside of medical speech recognition.
Broader impact
The implications of this work are significant for practitioners in the field of medical ASR, suggesting that traditional speech enhancement techniques may be counterproductive in certain contexts. This challenges long-held assumptions about preprocessing in ASR and encourages a reevaluation of enhancement methods in deployment scenarios. The findings could lead to more effective ASR systems in noisy environments, ultimately improving clinical documentation processes. This paper presents a critical examination of the effects of speech enhancement on modern ASR systems in medical contexts, revealing that such preprocessing can degrade performance rather than improve it. The comprehensive methodology and significant findings challenge existing paradigms in the field, highlighting the need for further investigation into the relationship between enhancement techniques and ASR performance.
Author of this paper? Send a correction.