MOSS-TTS Technical Report
Abstract
This technical report presents MOSS-TTS, a speech generation foundation model built on a scalable recipe: discrete audio tokens, autoregressive modeling, and large-scale pretraining. Built on MOSS-Audio-Tokenizer, a causal Transformer tokenizer that compresses 24 kHz audio to 12.5 fps with variable-bitrate RVQ and unified semantic-acoustic representations, we release two complementary generators: MOSS-TTS, which emphasizes structural simplicity, scalability, and long-context/control-oriented deployment, and MOSS-TTS-Local-Transformer, which introduces a frame-local autoregressive module for higher modeling efficiency, stronger speaker preservation, and a shorter time to first audio. Across multilingual and open-domain settings, MOSS-TTS supports zero-shot voice cloning, token-level duration control, phoneme-/pinyin-level pronunciation control, smooth code-switching, and stable long-form generation. This report summarizes the design, training recipe, and empirical characteristics of the released models.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of MOSS-TTS, a scalable speech generation model that emphasizes control and efficiency in audio synthesis. The technical contribution is significant, addressing current limitations in speech generation while providing a flexible framework for future research and application in the audio domain.
Methodology
The methodology presented in MOSS-TTS is well-structured, leveraging a causal Transformer tokenizer and autoregressive modeling to achieve efficient speech generation. The introduction of MOSS-Audio-Tokenizer for compressing audio and the dual generator architecture (MOSS-TTS and MOSS-TTS-Local-Transformer) demonstrates a thoughtful approach to scalability and control in speech synthesis. The focus on zero-shot voice cloning and token-level control adds significant value to the framework, indicating a robust understanding of current challenges in the field.
Results & evaluation
The paper outlines a comprehensive evaluation across multilingual and open-domain settings, which is essential for demonstrating the model's versatility. However, the lack of detailed quantitative results or comparisons with existing state-of-the-art models limits the assessment of its performance. The empirical characteristics are mentioned, but more rigorous benchmarking against established metrics would strengthen the claims of superiority.
Reproducibility
While the paper provides a clear design and training recipe, it lacks specific implementation details and code availability, which are critical for reproducibility. The absence of a project URL further complicates the ability for other researchers to replicate the results or build upon this work.
Limitations
The paper does not adequately address potential limitations of the MOSS-TTS framework, such as the computational resources required for training and inference or the potential biases in voice cloning across different demographics. Additionally, the evaluation metrics used for assessing model performance could be more thoroughly discussed.
Broader impact
MOSS-TTS has the potential to significantly impact various applications, including virtual assistants, content creation, and accessibility tools for individuals with speech impairments. The ability to perform zero-shot voice cloning and nuanced control over speech generation could lead to more personalized and engaging user experiences. The main contribution of this paper is the introduction of MOSS-TTS, a scalable speech generation model that emphasizes control and efficiency in audio synthesis. The technical contribution is significant, addressing current limitations in speech generation while providing a flexible framework for future research and application in the audio domain.
Author of this paper? Send a correction.