Improving Automatic Speech Recognition for Speakers Treated for Oral Cancer using Data Augmentation and LLM Error Correction
Abstract
In recent years, the performance of automatic speech recognition (ASR) systems has made considerable progress. Unfortunately, for people with speech impairments, such as people treated for oral cancer (OC), ASR performance is still lagging behind. The scarcity and variability of OC speech data makes development of ASR models for this type of speech difficult. In this work, we use data augmentation and large language model (LLM) error correction to mitigate this problem. We apply various augmentation techniques on a corpus of Dutch oral cancer speech to create synthetic data, and evaluate their effect on ASR performance. We finetune Whisper and Massively Multilingual Speech (MMS) models for each augmentation technique and observe, on average, an 8% relative decrease in Word Error Rate (WER) when including data created using text-to-speech (TTS). When employing LLMs for error correction, we see a further 21.4-26.2% relative decrease in WER for finetuned ASR models and a 10.0% relative decrease for non-finetuned models. Overall, we achieve a 40% relative WER decrease for Whisper and a 50% relative WER decrease for MMS, indicating that a combination of data augmentation and LLM correction is a viable strategy for the recognition of OC speech.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper lies in its innovative approach to enhancing automatic speech recognition for oral cancer patients through data augmentation and large language model error correction. This comprehensive analysis of the technical contributions, methodology, and significance to the field highlights the potential for improved communication technologies for individuals with speech impairments.
Methodology
The authors propose a comprehensive methodology that combines various data augmentation techniques (time stretching, speed perturbation, vocal tract length perturbation, voice conversion, and text-to-speech) to enhance ASR performance for oral cancer speech. The integration of LLMs for error correction is particularly innovative, providing a dual approach to improving recognition accuracy. The methodology is well-structured, with clear descriptions of each augmentation technique and its rationale, alongside a robust experimental design that includes both finetuning and error correction.
Results & evaluation
The experiments are thorough, utilizing a leave-one-speaker-out approach to evaluate the ASR models on a dataset specifically tailored for oral cancer speech. The performance metrics, primarily Word Error Rate (WER), are well-documented, showing significant improvements across various models and augmentation methods. The results demonstrate the effectiveness of TTS augmentation and LLM error correction, with detailed comparisons across different configurations, which adds to the reliability of the findings.
Reproducibility
The paper provides sufficient detail regarding the implementation of the augmentation methods and the ASR models used, including hyperparameters and experimental setups. However, the lack of publicly available code or datasets limits the reproducibility of the results. Future work could benefit from sharing these resources to enhance transparency and facilitate further research in this area.
Limitations
The study acknowledges several limitations, including the small dataset size and the ecological validity of the speech data, which may not generalize well to real-world scenarios. The reliance on a single augmentation method per experiment and the computational cost of the LLMs for error correction are also noted as potential drawbacks.
Broader impact
This research has significant implications for improving ASR systems for individuals with speech impairments, particularly those affected by oral cancer. By addressing a critical gap in ASR performance for pathological speech, the findings could lead to more accessible communication technologies for affected individuals, enhancing their quality of life. The combination of data augmentation and LLMs may also inspire future research in other domains of speech recognition. The main contribution of this paper lies in its innovative approach to enhancing automatic speech recognition for oral cancer patients through data augmentation and large language model error correction. This comprehensive analysis of the technical contributions, methodology, and significance to the field highlights the potential for improved communication technologies for individuals with speech impairments.
Author of this paper? Send a correction.