SynTTS-Commands: A Public Dataset for On-Device KWS via TTS-Synthesized Multilingual Speech
Abstract
The development of high-performance, on-device keyword spotting (KWS) systems for ultra-low-power hardware is critically constrained by the scarcity of specialized, multi-command training datasets. Traditional data collection through human recording is costly, slow, and lacks scalability. This paper introduces SYNTTS-COMMANDS, a novel, multilingual voice command dataset entirely generated using state-of-the-art Text-to-Speech (TTS) synthesis. By leveraging the CosyVoice 2 model and speaker embeddings from public corpora, we created a scalable collection of English and Chinese commands. Extensive benchmarking across a range of efficient acoustic models demonstrates that our synthetic dataset enables exceptional accuracy, achieving up to 99.5\% on English and 98\% on Chinese command recognition. These results robustly validate that synthetic speech can effectively replace human-recorded audio for training KWS classifiers. Our work directly addresses the data bottleneck in TinyML, providing a practical, scalable foundation for building private, low-latency, and energy-efficient voice interfaces on resource-constrained edge devices.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents SYNTTS-COMMANDS, a novel multilingual voice command dataset generated using TTS technology, demonstrating that synthetic speech can effectively replace human-recorded audio for training KWS classifiers. The work addresses a critical data bottleneck in the field, paving the way for more efficient and inclusive voice-enabled applications on resource-constrained devices.
Methodology
The methodology is robust, leveraging state-of-the-art TTS technology to generate a synthetic dataset that addresses the critical bottleneck in KWS systems. The use of the CosyVoice 2 model and speaker embeddings from established corpora is a strong choice, ensuring high-quality output. The rigorous quality validation process, including automated transcription and manual verification, enhances the credibility of the dataset. However, the methodology could benefit from a more detailed explanation of the selection criteria for the commands and the specific acoustic models used in the benchmarking.
Results & evaluation
The experimental evaluation is comprehensive, demonstrating the dataset's effectiveness through extensive benchmarking across various acoustic models. The results, showing high accuracy rates for both English and Chinese commands, validate the dataset's utility. However, the paper could improve by including comparisons with existing datasets and discussing potential biases in the synthetic data.
Reproducibility
The paper outlines a clear dataset construction process and evaluation methodology, which supports reproducibility. However, the lack of a publicly available dataset or code repository limits the ability for other researchers to replicate the study fully. Providing access to the dataset and the models used would significantly enhance reproducibility.
Limitations
The primary limitation is the dataset's focus on in-class command recognition, which does not account for out-of-vocabulary inputs or environmental noise. This could affect real-world applicability. Additionally, while the dataset is multilingual, it currently only includes English and Chinese, which may limit its use in diverse linguistic contexts.
Broader impact
The introduction of a scalable, synthetic voice command dataset has significant implications for the development of on-device AI and TinyML applications. By reducing reliance on human-recorded data, it lowers costs and accelerates the development of multilingual voice interfaces, promoting inclusivity in voice technology. The potential for future expansion into other languages and command types could further enhance its impact. This paper presents SYNTTS-COMMANDS, a novel multilingual voice command dataset generated using TTS technology, demonstrating that synthetic speech can effectively replace human-recorded audio for training KWS classifiers. The work addresses a critical data bottleneck in the field, paving the way for more efficient and inclusive voice-enabled applications on resource-constrained devices.
Author of this paper? Send a correction.