Few-Shot Accent Synthesis for ASR with LLM-Guided Phoneme Editing
Abstract
Accented automatic speech recognition (ASR) often degrades due to the limited availability of accented training data. Prior work has explored accent modeling in low-resource settings, but existing approaches typically require minutes to hours of labeled speech, which may still be impractical for truly scarce accent scenarios. We propose a pipeline that adapts a text-to-speech (TTS) decoder to a target-accent speaker using fewer than ten reference utterances and employs large language model (LLM)-based phoneme editing to generate accent-conditioned pronunciations. The resulting synthetic speech is used to fine-tune a self-supervised ASR model. Experiments demonstrate consistent word error rate (WER) reductions on real accented speech, including cross-speaker evaluation and ultra-low data regimes. A matched-rate random phoneme baseline shows that phoneme-space perturbation itself is a strong form of augmentation, while LLM-guided edits provide additional gains through accent-conditioned structure.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the development of a few-shot accent synthesis pipeline that leverages LLM-guided phoneme editing to improve ASR performance in low-resource settings. This innovative approach not only addresses the challenge of accent adaptation but also demonstrates the effectiveness of combining TTS and ASR technologies to enhance speech recognition across diverse accents.
Methodology
The proposed methodology effectively combines few-shot learning with LLM-guided phoneme editing to address the challenge of accent adaptation in ASR systems. The approach is innovative in its use of a phoneme-conditioned TTS model and the integration of LLMs for phoneme editing, which allows for accent-specific pronunciation adjustments while maintaining prosodic alignment. The system's architecture is well-defined, and the use of a matched-rate random phoneme baseline provides a strong comparative framework to evaluate the effectiveness of the LLM-guided edits.
Results & evaluation
The experiments are comprehensive, evaluating the proposed method across multiple accents (Indian and Korean English) and demonstrating significant improvements in WER through synthetic data generation. The paper provides a clear experimental setup, including detailed descriptions of the datasets, evaluation metrics, and results. The findings indicate that the proposed method not only enhances ASR performance in low-resource scenarios but also shows potential for cross-speaker generalization, which is a critical aspect of practical ASR applications.
Reproducibility
The paper includes sufficient implementation details, including training configurations, feature extraction methods, and evaluation protocols, which support reproducibility. However, the absence of a public code repository limits the ease with which other researchers can replicate the results. The authors should consider releasing their code and models to enhance reproducibility.
Limitations
One notable limitation is that the system inherits prosody from the source speech rather than modeling accent-specific prosodic variations, which may restrict the fidelity of the synthesized speech. Additionally, the adaptation is limited to a single reference speaker, which could affect the generalizability of the results across different speakers and accents. Future work should address these limitations by exploring multi-speaker accent generation and explicit prosody modeling.
Broader impact
The research has significant implications for improving ASR systems in diverse linguistic contexts, particularly for underrepresented accents. By enabling effective accent adaptation with minimal data, this work can contribute to more inclusive speech technologies that better serve global populations. The potential applications extend to various domains, including voice assistants, transcription services, and accessibility tools, enhancing communication for speakers of different accents. The main contribution of this paper is the development of a few-shot accent synthesis pipeline that leverages LLM-guided phoneme editing to improve ASR performance in low-resource settings. This innovative approach not only addresses the challenge of accent adaptation but also demonstrates the effectiveness of combining TTS and ASR technologies to enhance speech recognition across diverse accents.
Author of this paper? Send a correction.