UrduSpeech: A 156-Hour Urdu Speech Corpus with 12-Dimension Paralinguistic Annotations
Abstract
Despite 230 million speakers, Urdu remains critically under-resourced in speech technology. We introduce UrduSpeech: a large high-fidelity Urdu corpus comprising 156 hours of audio with 12-dimension paralinguistic metadata, encompassing US-Std, US-CS, US-EngPk. To address Right-to-Left script constraints and frequent code-switching, we developed UrduSpeech, a LLM-driven pipeline to curate data across 12 diverse categories, including news, drama, and rare literary forms like Bait-Bazi. We also release a 9-hour US-Benchmark set, manually corrected by native annotators to serve as a standard. Human quality assessment of the primary 156-hour corpus yielded a Mean Opinion Score (MOS) of 4.6 (std = 0.7) with inter-rater reliability confirmed by a 0.68 Cohen's Kappa, validating our curation pipeline's 97.6% confidence score. The corpus maintains a 60-40 gender balance across 71,792 utterances. Our work represents a significant leap toward linguistic inclusivity in global AI. The corpus and code are open-sourced, and a demo page is available.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of the UrduSpeech corpus, a comprehensive resource for Urdu speech technology that includes high-fidelity audio and extensive paralinguistic annotations. This work significantly enhances the landscape of speech resources for under-resourced languages, addressing critical gaps in existing datasets and methodologies.
Methodology
The methodology is robust, leveraging a multi-stage pipeline for data curation that addresses the unique challenges of Urdu speech, including RTL script and code-switching. The authors employed advanced techniques such as speaker diarization and noise removal, ensuring high-quality audio segments. The integration of 12-dimensional paralinguistic annotations is a significant enhancement, allowing for detailed analysis of emotional and vocal characteristics. The use of generative models for transcription and annotation, along with a rigorous human-centric validation framework, further strengthens the methodology.
Results & evaluation
The experiments are comprehensive, with a clear focus on establishing a baseline for Urdu ASR and TTS. The authors conducted a pilot study and a thorough evaluation of various transcription models, providing detailed comparisons and insights into their performance. The Mean Opinion Score (MOS) and inter-rater reliability metrics demonstrate the corpus's high fidelity and reliability, which are crucial for future research and applications.
Reproducibility
The paper outlines the data collection and preprocessing steps in detail, which aids in reproducibility. However, the absence of a publicly accessible project URL limits the ability for others to directly replicate the study. The authors mention open-sourcing the corpus and code, which is a positive aspect for reproducibility.
Limitations
The paper acknowledges several limitations, including potential over-segmentation in speaker diarization and the presence of background noise in some audio segments. While the authors have made efforts to validate the gender distribution and speaker IDs, ongoing work is needed to ensure absolute compliance. The reliance on automated systems for initial processing may introduce errors that require manual correction.
Broader impact
The UrduSpeech corpus represents a significant advancement in the field of speech technology for under-resourced languages, particularly Urdu. By providing a high-quality, diverse dataset, this work has the potential to enhance the performance of ASR and TTS systems for Urdu and related dialects, fostering linguistic inclusivity in AI applications. The integration of paralinguistic metadata opens new avenues for research in affective computing and speaker profiling. The main contribution of this paper is the introduction of the UrduSpeech corpus, a comprehensive resource for Urdu speech technology that includes high-fidelity audio and extensive paralinguistic annotations. This work significantly enhances the landscape of speech resources for under-resourced languages, addressing critical gaps in existing datasets and methodologies.
Author of this paper? Send a correction.