REAL: Reward Modeling through Evidence-Linked Analysis of Lifelikeness in Generated Speech
Abstract
As modern text-to-speech systems become increasingly intelligible and natural, evaluation is shifting from detecting gross synthesis failures to identifying subtle departures from lifelike human delivery. Because large-scale listening tests are costly and slow, speech reward models can serve as scalable automatic judges for comparing generated utterances, curating data, and supporting iterative TTS development. Existing speech reward models can predict scores or pairwise preferences and may generate natural-language rationales, but their supervision is typically defined at the utterance or pair level. The connection between local acoustic phenomena and the final preference therefore remains implicit. We introduce REAL, a framework for Reward modeling through Evidence-linked Analysis of Lifelikeness in generated speech. Its central idea is to supervise not only the final pairwise preference but also a structured account of its supporting observations, linking local audio spans to perceptual phenomena, affected dimensions, and the overall judgment. Specifically, REAL-Jury constructs these evidence-linked reports through timestamp alignment, multi-model analysis, and dispute-oriented deliberation. REAL-Judge learns to generate them from paired audio inputs. REAL-Eval assesses preference prediction in two tiers, from salient perceptual differences to subtle distinctions among strong TTS systems. Instantiated with MiMo-Audio, REAL-Judge improves pairwise preference accuracy over its backbone by 16.01 and 11.41 percentage points on the two tiers, respectively, and outperforms the open-source speech reward models SpeechJudge and UniSRM on both tiers.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a novel framework for speech reward modeling that explicitly links local acoustic evidence to global preference judgments through a multi-model deliberation process. By constructing structured, timestamp-aligned supervision signals and distilling them into a single model, REAL-Judge achieves state-of-the-art performance in both salient defect detection and subtle preference discrimination, offering a scalable and interpretable alternative to human evaluation for TTS systems.
Methodology
The paper proposes REAL, a framework for speech reward modeling that explicitly supervises the link between local acoustic evidence and global preference judgments. The methodology is structured into three components: REAL-Jury, which uses a multi-model orchestration (coordinator + specialized models) to generate timestamp-aligned, dispute-resolved evidence reports; REAL-Judge, which distills this structured supervision into a single audio-language model using LoRA; and REAL-Eval, a two-tier benchmark. The core technical contribution is the "evidence-linked" supervision signal, which forces the model to ground its preference in specific time spans and perceptual phenomena rather than relying on implicit global features. The use of explicit time-marker embeddings in the audio representation is a practical and effective design choice for temporal grounding. The deliberation process in REAL-Jury is a sophisticated approach to data construction, leveraging model disagreement to refine evidence quality.
Results & evaluation
The experimental setup is robust, utilizing a large training set (11,947 pairs) constructed via the proposed pipeline. The evaluation is well-designed with two distinct tiers: Standard (salient defects) and Fine (subtle differences among strong systems). The results show significant improvements over the backbone model (MiMo-Audio) and outperform strong baselines like SpeechJudge and UniSRM. The ablation studies effectively demonstrate the contribution of time markers and the deliberation process. The correlation analysis (Pearson r=0.96) between predicted and human win rates is a strong indicator of the model's utility for system-level evaluation.
Reproducibility
The paper provides detailed hyperparameters for training (LoRA rank, learning rate, etc.) and describes the architecture of the jury. However, the specific prompts used for the coordination model and the exact criteria for "dispute resolution" are not fully detailed in the provided text, which may limit exact reproduction of the data construction pipeline. The code and weights are promised to be released, which is a positive sign.
Limitations
The work is currently limited to Chinese speech, which restricts its immediate generalizability. The reliance on a complex multi-model jury for data construction makes the training data generation process computationally expensive and potentially difficult to scale to other languages or domains without significant engineering effort. The evaluation, while strong, is based on a relatively small number of pairs (400 and 207) for the fine-grained tier, which may affect statistical significance.
Broader impact
This work has significant potential for automating the evaluation of TTS systems, reducing the need for costly human listening tests. The evidence-linked approach could be extended to other audio tasks, such as music generation evaluation or audio captioning, where grounding in specific temporal segments is crucial. The framework offers a path towards more interpretable and trustworthy AI judges for audio content. The paper introduces a novel framework for speech reward modeling that explicitly links local acoustic evidence to global preference judgments through a multi-model deliberation process. By constructing structured, timestamp-aligned supervision signals and distilling them into a single model, REAL-Judge achieves state-of-the-art performance in both salient defect detection and subtle preference discrimination, offering a scalable and interpretable alternative to human evaluation for TTS systems.
Author of this paper? Send a correction.