ParaStyleTTS: Toward Efficient and Robust Paralinguistic Style Control for Expressive Text-to-Speech Generation
Abstract
Controlling speaking style in text-to-speech (TTS) systems has become a growing focus in both academia and industry. While many existing approaches rely on reference audio to guide style generation, such methods are often impractical due to privacy concerns and limited accessibility. More recently, large language models (LLMs) have been used to control speaking style through natural language prompts; however, their high computational cost, lack of interpretability, and sensitivity to prompt phrasing limit their applicability in real-time and resource-constrained environments. In this work, we propose ParaStyleTTS, a lightweight and interpretable TTS framework that enables expressive style control from text prompts alone. ParaStyleTTS features a novel two-level style adaptation architecture that separates prosodic and paralinguistic speech style modeling. It allows fine-grained and robust control over factors such as emotion, gender, and age. Unlike LLM-based methods, ParaStyleTTS maintains consistent style realization across varied prompt formulations and is well-suited for real-world applications, including on-device and low-resource deployment. Experimental results show that ParaStyleTTS generates high-quality speech with performance comparable to state-of-the-art LLM-based systems while being 30x faster, using 8x fewer parameters, and requiring 2.5x less CUDA memory. Moreover, ParaStyleTTS exhibits superior robustness and controllability over paralinguistic speaking styles, providing a practical and efficient solution for style-controllable text-to-speech generation. Demo can be found at https://parastyletts.github.io/ParaStyleTTS_Demo/. Code can be found at https://github.com/haoweilou/ParaStyleTTS.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
ParaStyleTTS represents a significant contribution to the field of text-to-speech generation, offering an innovative approach to style control that balances efficiency and expressiveness. The methodology's emphasis on separating prosodic and paralinguistic features, combined with a robust experimental evaluation, positions this work as a valuable advancement in the ongoing development of TTS technologies.
Methodology
The proposed methodology of ParaStyleTTS is innovative, utilizing a two-level style adaptation architecture that effectively separates prosodic and paralinguistic features, allowing for fine-grained control over speech generation. The model's end-to-end design is particularly noteworthy, as it eliminates the need for external vocoders and achieves high-quality speech synthesis directly from text prompts. The use of a lightweight architecture that is computationally efficient is a significant advancement over existing LLM-based methods, which are often resource-intensive and less interpretable.
Results & evaluation
The experimental evaluation is robust, employing a comprehensive multilingual dataset that includes diverse speech samples across different styles and emotions. The authors conducted both objective metrics and subjective evaluations (MOS tests) to assess intelligibility and naturalness, providing a well-rounded view of the model's performance. The results demonstrate that ParaStyleTTS achieves competitive performance compared to state-of-the-art models while significantly improving efficiency, which is crucial for real-world applications.
Reproducibility
The paper includes sufficient details regarding the architecture, training procedures, and datasets used, which enhances reproducibility. The authors provide links to the demo and code repositories, allowing other researchers to replicate their findings and build upon their work. However, the lack of specific hyperparameters and training configurations could be a minor barrier to complete reproducibility.
Limitations
While ParaStyleTTS shows strong performance, it still lags slightly behind LLM-based models in overall intelligibility and subjective naturalness. The model currently supports only three paralinguistic styles, which limits its applicability. Additionally, the authors acknowledge that expanding the training dataset and the range of controllable styles is a future direction, indicating that there is room for improvement.
Broader impact
The advancements presented in ParaStyleTTS have significant implications for the development of more efficient and expressive TTS systems. Its ability to operate effectively in resource-constrained environments makes it suitable for a wide range of applications, including virtual assistants, accessibility tools, and interactive storytelling. The model's robustness to prompt variations also enhances its usability in real-world scenarios, where user input may vary significantly. ParaStyleTTS represents a significant contribution to the field of text-to-speech generation, offering an innovative approach to style control that balances efficiency and expressiveness. The methodology's emphasis on separating prosodic and paralinguistic features, combined with a robust experimental evaluation, positions this work as a valuable advancement in the ongoing development of TTS technologies.
Author of this paper? Send a correction.