DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancement
Abstract
Dysarthric speech exhibits abnormal prosody and significant speaker variability, presenting persistent challenges for automatic speech recognition (ASR). While text-to-speech (TTS)-based data augmentation has shown potential, existing methods often fail to accurately model the pathological rhythm and acoustic style of dysarthric speech. To address this, we propose DARS, a dysarthria-aware rhythm-style synthesis framework based on the Matcha-TTS architecture. DARS incorporates a multi-stage rhythm predictor optimized by contrastive preferences between normal and dysarthric speech, along with a dysarthric-style conditional flow matching mechanism, jointly enhancing temporal rhythm reconstruction and pathological acoustic style simulation. Experiments on the TORGO dataset demonstrate that DARS achieves a Mean Cepstral Distortion (MCD) of 4.29, closely approximating real dysarthric speech. Adapting a Whisper-based ASR system with synthetic dysarthric speech from DARS achieves a 54.22% relative reduction in word error rate (WER) compared to state-of-the-art methods, demonstrating the framework's effectiveness in enhancing recognition performance.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the development of the DARS framework, which effectively synthesizes dysarthric speech to enhance automatic speech recognition performance, addressing a critical gap in assistive technology for individuals with speech impairments. The combination of innovative methodologies and rigorous experimental validation positions this work as a significant advancement in the field of speech synthesis and recognition.
Methodology
The DARS framework introduces innovative mechanisms for synthesizing dysarthric speech, specifically a multi-stage rhythm predictor and a dysarthria-aware conditional flow matching mechanism. The use of contrastive preference optimization to guide the rhythm predictor is particularly novel, as it directly addresses the variability in dysarthric speech patterns. The integration of pause modeling and acoustic style vectors enhances the synthesis quality, making the approach well-suited for the complexities of dysarthric speech.
Results & evaluation
The paper presents a thorough experimental evaluation using the TORGO dataset, demonstrating the effectiveness of the DARS framework in enhancing ASR performance. The reported results, including a 54.22% relative reduction in WER, indicate significant improvements over existing methods. The experiments are well-structured, comparing multiple training strategies and adaptation techniques, which adds robustness to the findings.
Reproducibility
While the paper provides a detailed description of the methodology and experimental setup, the absence of URLs for code or demo pages limits reproducibility. Clearer documentation or supplementary materials would enhance the ability for others to replicate the results.
Limitations
The study relies on a limited dataset (TORGO), which may affect the generalizability of the results. Additionally, while the framework shows promise, the performance on more diverse dysarthric speech samples and real-world scenarios remains to be validated.
Broader impact
The DARS framework has the potential to significantly improve communication aids for individuals with dysarthria, enhancing their quality of life. By improving ASR systems' ability to recognize dysarthric speech, this research could facilitate better interaction and accessibility for affected individuals in various settings. The main contribution of this paper is the development of the DARS framework, which effectively synthesizes dysarthric speech to enhance automatic speech recognition performance, addressing a critical gap in assistive technology for individuals with speech impairments. The combination of innovative methodologies and rigorous experimental validation positions this work as a significant advancement in the field of speech synthesis and recognition.
Author of this paper? Send a correction.