Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Masking
Abstract
Pre-trained transformer-based models have significantly advanced automatic speech recognition (ASR), yet they remain sensitive to accent and dialectal variations, resulting in elevated word error rates (WER) in linguistically diverse languages such as English and Persian. To address this challenge, we propose an accent-invariant ASR framework that integrates accent and dialect classification into the recognition pipeline. Our approach involves training a spectrogram-based classifier to capture accent-specific cues, masking the regions most influential to its predictions, and using the masked spectrograms for data augmentation. This enhances the robustness of ASR models against accent variability. We evaluate the method using both English and Persian speech. For Persian, we introduce a newly collected dataset spanning multiple regional accents, establishing the first systematic benchmark for accent variation in Persian ASR that fills a critical gap in multilingual speech research and provides a foundation for future studies on low-resource, linguistically diverse languages. Experimental results with the Whisper model demonstrate that our masking and augmentation strategy yields substantial WER reductions in both English and Persian settings, confirming the effectiveness of the approach. This research advances the development of multilingual ASR systems that are resilient to accent and dialect diversity. Code and dataset are publicly available at: https://github.com/MH-Sameti/Accent_invariant_ASR
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a novel approach to accent-invariant ASR that integrates accent classification into the recognition pipeline, significantly advancing the field of multilingual speech recognition. The methodology and experimental results demonstrate the potential for improved ASR performance in linguistically diverse settings, particularly for underrepresented languages like Persian.
Methodology
The proposed methodology integrates accent and dialect classification into the ASR pipeline, which is innovative in addressing the challenge of accent variability. The use of a spectrogram-based classifier to identify accent-specific cues and the subsequent masking of influential regions for data augmentation is a novel approach that enhances model robustness. However, the paper could benefit from a more detailed explanation of the classifier architecture and the specific masking techniques employed.
Results & evaluation
The experiments are well-structured, utilizing both English and Persian datasets, which adds to the robustness of the findings. The introduction of a new dataset for Persian accents is a significant contribution, providing a systematic benchmark for future research. The results indicate substantial reductions in WER, demonstrating the effectiveness of the proposed method. However, a comparison with existing state-of-the-art methods would strengthen the evaluation.
Reproducibility
The authors have provided a link to the code and dataset, which is essential for reproducibility. However, the paper lacks detailed implementation specifics, such as hyperparameter settings and training procedures, which could hinder full reproducibility by other researchers.
Limitations
One limitation is the focus on only two languages, which may not generalize well to other linguistically diverse languages. Additionally, the paper does not address the potential computational overhead introduced by the accent classification step, which could impact real-time ASR applications.
Broader impact
The research has significant implications for the development of multilingual ASR systems, particularly in low-resource languages. By addressing accent and dialect diversity, this work contributes to making ASR technology more inclusive and effective across different linguistic backgrounds. The introduction of a new dataset for Persian accents also opens avenues for further research in this area. This paper presents a novel approach to accent-invariant ASR that integrates accent classification into the recognition pipeline, significantly advancing the field of multilingual speech recognition. The methodology and experimental results demonstrate the potential for improved ASR performance in linguistically diverse settings, particularly for underrepresented languages like Persian.
Author of this paper? Send a correction.