Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis
Abstract
While large language model (LLM)-based text-to-speech (TTS) systems have achieved high-quality speech synthesis, most existing systems focus on English and Chinese. Japanese, however, remains under-explored, and its unique linguistic challenges, such as widespread context-dependent kanji polyphony, have yet to be adequately tackled. Here we introduce Sarashina2.2-TTS (https://github.com/sbintuitions/sarashina2.2-tts), a Japanese-centric LLM-TTS system that tackles these challenges through a dual approach: data strategy and evaluation methodology. First, we scale training to approximately 361k hours of speech, incorporating a balanced mix of Japanese and English data. Furthermore, we design a targeted data augmentation pipeline covering all 2,136 Joyo (regular-use) kanji designated by Japan's Agency for Cultural Affairs to efficiently address kanji polyphony disambiguation. Second, we introduce the Joyo Kanji Yomi Benchmark (https://github.com/sbintuitions/JoyoKanji-Yomi-Benchmark), covering all 2,136 Joyo kanji and their 4,378 readings. Alongside this benchmark, we propose Kana-CER, a metric that compares synthesized speech against reference readings in the kana space, eliminating orthographic variations to directly measure pronunciation correctness. Experiments demonstrate that our targeted data augmentation significantly improves reading accuracy. Overall, Sarashina2.2-TTS achieves state-of-the-art kanji-level reading accuracy and matches top baselines on general sentence-level pronunciation, while delivering the highest speaker similarity in zero-shot Japanese speech synthesis. Furthermore, cross-lingual evaluation reveals that Sarashina2.2-TTS is the only system that maintains stable Japanese pronunciation regardless of the prompt language, confirming that our balanced training approach improves cross-lingual robustness.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
Sarashina2.2-TTS makes a significant contribution to Japanese speech synthesis by introducing a targeted synthetic data augmentation pipeline for kanji polyphony and a novel kana-based evaluation metric, achieving state-of-the-art reading accuracy and cross-lingual robustness.
Methodology
The paper proposes a comprehensive data-centric strategy to address the specific linguistic challenge of kanji polyphony in Japanese TTS. The core methodological contribution is the construction of a massive 361k-hour multilingual dataset with a balanced Japanese-English ratio, which is unusually large for open-source Japanese TTS. The most novel technical component is the "Pronunciation Steering" (PronSteering) mechanism, which uses special tokens to inject explicit kana readings and pitch-accent tags into the LLM context. This is leveraged in a targeted synthetic data generation pipeline to cover rare kanji readings. The authors also introduce a novel evaluation metric, Kana-CER, which operates in the phonological (kana) space rather than the orthographic (kanji) space to eliminate errors caused by Japanese orthographic variation. The architecture itself (S3Tokenizer + LLM + Flow Matching) is derivative of existing LLM-TTS systems (like CosyVoice), but the data engineering and evaluation framework are distinct and highly relevant to the subfield. EXPERIMENTAL_EVALUTION: The experimental evaluation is rigorous and well-designed for the specific problem. The authors introduce the "Joyo Kanji Yomi Benchmark," a human-verified dataset covering all 2,136 Joyo kanji and 4,378 readings, which fills a critical gap in the field. Results show state-of-the-art performance on this benchmark, significantly outperforming baselines like Qwen3-TTS and FishAudio S1-mini in kanji-level accuracy. The cross-lingual robustness experiments are particularly compelling, demonstrating that the balanced training data prevents the degradation of Japanese pronunciation when prompted with non-Japanese speech, a common failure mode in multilingual models. The use of standard CER vs. Kana-CER effectively highlights the limitations of existing evaluation metrics for Japanese.
Reproducibility
The paper provides code and model weights for the TTS system and the benchmark. The detailed description of the PronSteering tokenization and the synthetic data generation pipeline enhances reproducibility. However, the exact sources of the 361k hours of data are not fully enumerated (only domains are described), which is typical for large-scale proprietary data curation but limits full reproducibility of the data distribution. The Kana-ASR model is also released, aiding in the reproducibility of the evaluation metric.
Limitations
The PronSteering capability is explicitly stated as *not* included in the open-source release; users only get the model trained with synthetic data generated by it. This limits the immediate utility of the method for users who want to control pronunciation dynamically. The reliance on LLM-generated sentences for the synthetic data and benchmark introduces potential biases or unnatural phrasing, although human verification mitigates this. The Kana-ASR model, while effective, may struggle with highly expressive or colloquial speech, as noted by the authors.
Broader impact
This work significantly advances the state of Japanese TTS, a language often underserved compared to English and Chinese. By providing a standardized benchmark and evaluation metric, it facilitates fairer comparisons and drives progress in handling complex orthographic-to-phonological mappings. The balanced training strategy offers insights for improving cross-lingual robustness in multilingual models. The open-source release of the benchmark and tools will likely spur further research into low-resource language handling and polyphony disambiguation. Sarashina2.2-TTS makes a significant contribution to Japanese speech synthesis by introducing a targeted synthetic data augmentation pipeline for kanji polyphony and a novel kana-based evaluation metric, achieving state-of-the-art reading accuracy and cross-lingual robustness.
Author of this paper? Send a correction.