Preference Optimization for Non-Verbal Vocalization Synthesis
Abstract
Non-verbal vocalizations (NVs), such as laughter, coughs, and sighs, are essential for expressive TTS, but the effectiveness of preference optimization for NV generation remains poorly understood. We systematically study preference optimization for NV-capable TTS, focusing on preference signals, preference-pair construction, and DPO-based optimization objectives. We formulate an NV-aware character error rate (NV-CER) by treating NV tags as distinct output symbols and computing a weighted pinyin-based CER over both verbal and non-verbal content, enabling controllable optimization of NV realization without modifying the underlying optimization algorithm. Experiments on Emilia-NV and the augmented NV-Bench covering 18 NV types reveal how different design choices affect NV realization and lexical fidelity, and establish an effective setup using standard DPO. Objective, LLM-based, and human evaluations provide converging evidence for our findings, offering practical insights into NV-aware post-training for expressive TTS.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a systematic evaluation of DPO for non-verbal vocalization synthesis in TTS, introducing an NV-aware CER metric and demonstrating that NV-specific preference signals improve NV accuracy and overall synthesis quality over standard baselines.
Methodology
The paper proposes a systematic study of Direct Preference Optimization (DPO) for Non-Verbal Vocalization (NV) synthesis in Text-to-Speech (TTS) systems. The core methodological contribution is the formulation of an NV-aware Character Error Rate (NV-CER) that treats NV tags as distinct symbols and computes a weighted edit distance over pinyin and NV tokens. This metric is used to construct preference pairs (preferred vs. rejected) for DPO. The authors also introduce a weighting mechanism ($w_{NV}$) to control the trade-off between NV realization and lexical fidelity. While the application of DPO to TTS is not entirely new, its systematic application and evaluation specifically for the nuanced task of NV generation, along with the specific NV-CER metric, provides a structured approach to a previously under-explored area. The methodology is sound but largely incremental, relying on standard DPO formulations and existing ASR models (SenseVoice) for preference signals.
Results & evaluation
The experiments are conducted on the Emilia-NV dataset and the NV-Bench. The authors evaluate various preference signals (ASR CER, NV-CER, NV-CER+UTMOS) and loss formulations (DPO, DPO+SFT). Results show that NV-CER-based preference signals outperform conventional ASR-CER and baseline SFT models in both objective metrics (NV-CER, PCER, UTMOS, DNSMOS) and LLM-based/human evaluations. The ablation studies on preference pair construction (Synthetic vs. Ground Truth) and NV-weight tuning provide valuable insights. However, the improvements in perceptual quality (UTMOS/DNSMOS) are modest, and the human evaluation shows high tie rates, suggesting that while NV accuracy improves, naturalness gains are less pronounced. The use of LLM-based evaluation adds a modern layer of assessment, though it correlates with objective metrics.
Reproducibility
The paper provides sufficient implementation details, including the base model (CosyVoice2-0.5B), training hyperparameters (learning rates, epochs, batch sizes), and the specific NV-ASR model used (SenseVoice-Small fine-tuned). The dataset (Emilia-NV) and benchmark (NV-Bench) are publicly available or can be constructed as described. The code is not explicitly linked, but the methodology is clear enough for reproduction by researchers familiar with LLM-based TTS systems.
Limitations
The primary limitation is that the preference signal relies entirely on an ASR model, which may not perfectly correlate with human perception of NV naturalness or correctness, especially for ambiguous NVs. The paper acknowledges that the base model rarely omits NVs, making "no-NV" a weak negative example, which limits the diversity of preference pairs. Furthermore, the improvements in perceptual quality are marginal, and the method does not fundamentally change the underlying TTS architecture, relying instead on post-training. The reliance on pinyin for Mandarin also limits immediate generalizability to other languages without adaptation.
Broader impact
This work contributes to the field of expressive TTS by providing practical guidelines for incorporating NVs into preference optimization pipelines. It helps bridge the gap between lexical TTS and more human-like, expressive speech synthesis. The NV-CER metric offers a standardized way to evaluate NV generation, which could facilitate future research in this area. The findings are relevant for developers of conversational AI agents and virtual assistants requiring more natural interaction capabilities. This paper presents a systematic evaluation of DPO for non-verbal vocalization synthesis in TTS, introducing an NV-aware CER metric and demonstrating that NV-specific preference signals improve NV accuracy and overall synthesis quality over standard baselines.
Author of this paper? Send a correction.