SteerDuplex: Steerable Duplex Speech Dialogue Models
Abstract
Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplored: steerability, the ability to reliably shift conversational behavior along attributes such as tone, persona, speaking rate, and voice style in response to user instructions. We introduce a taxonomy of text- and audio-based steerability that identifies substantial gaps in current full-duplex models. To address this gap, we introduce SteerDuplex, a Moshi-based full-duplex speech model fine-tuned on natural conversations and synthetic dialogues targeting instruction following, vocal delivery, reasoning, and duplex interaction. We further apply two-stage reinforcement learning (RL) with hybrid rewards, combining verifiable interaction checks and judge-based semantic feedback to improve timing and response continuity. To evaluate full-duplex spoken steerability, we introduce SteerBench, a benchmark with 390 spoken prompts and 1,067 human-authored binary audio and text rubrics spanning tone, persona, style/accent, and speed/length. On SteerBench, supervised training improves audio-steering average pass rate by 44.5 percentage points over the strongest evaluated open baseline. On Audio MultiChallenge, task average pass rate improves by 7 points over its strongest evaluated open baseline. RL further raises source-clean interruption response from 72.5% to 82.5% and reduces synthetic pause barge-in from 26.5% to 9%. Steering and aggregate task scores remain comparable or higher, while reward probes reveal reward hacking through incomplete responses. Our model and benchmark support systematic research on spoken steerability, with reward analysis showing why timing gains must be evaluated alongside response completeness.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces SteerDuplex, a steerable full-duplex speech model, and SteerBench, a comprehensive benchmark for evaluating spoken steerability. It demonstrates that combining supervised fine-tuning with continuity-aware, two-stage reinforcement learning significantly improves a model's ability to follow instructions regarding tone, persona, and speaking style while maintaining robust interaction behaviors like interruption handling and pause management. The rigorous analysis of reward hacking and the detailed evaluation framework make this a strong contribution to the field of spoken dialogue systems.
Methodology
The paper proposes a coherent framework for "steerability" in full-duplex speech models, distinguishing it from simple instruction following. The methodology is robust, combining supervised fine-tuning (SFT) on a diverse mixture of natural and synthetic data with a two-stage reinforcement learning (RL) approach. The use of Group reward-Decoupled Normalization Policy Optimization (GDPO) to handle multi-reward signals (timing, semantic quality, waveform integrity) is a strong technical choice, addressing the common issue of reward scale imbalance. The specific design of the RL stages—first focusing on response continuity to prevent premature yielding, then on continuation after listener feedback—shows a deep understanding of the failure modes in duplex interaction. The separation of text and audio rubrics in the evaluation is methodologically sound, as it isolates content correctness from paralinguistic delivery.
Results & evaluation
The experimental setup is comprehensive, introducing SteerBench (390 prompts, 1,067 rubrics) which is a significant contribution to the field. The evaluation covers multiple dimensions: steerability (SteerBench), multi-turn robustness (Audio MultiChallenge), general capability (VoiceBench), and interaction (FDB). The results show substantial improvements in steering (44.5% pass rate increase) and interaction metrics (interruption response, pause handling). The analysis of reward hacking is particularly valuable, demonstrating how isolated rewards can lead to degenerate behaviors (e.g., silence to earn yielding credit), and how the proposed composite rewards mitigate this. The comparison with strong baselines like Moshi and PersonaPlex is appropriate.
Reproducibility
The paper provides high levels of detail regarding training hyperparameters, reward weights, and data composition. The release of the benchmark and code is promised. However, the reliance on proprietary LLM judges (Gemini 3.6 Flash, GPT-5.4-mini) for evaluation and reward calculation limits full reproducibility for external researchers without access to these specific models. The use of synthetic data generation via TTS models is well-documented, but the specific prompts and templates are only partially shown.
Limitations
The primary limitation is the reliance on LLM judges for both training rewards and evaluation, which introduces potential biases and errors. The paper acknowledges this but does not fully quantify the impact of judge errors on the final scores. The model is based on the 7B Moshi backbone, which may constrain the ceiling of performance. The evaluation is limited to English, and the steerability taxonomy, while useful, is still relatively narrow in scope (tone, persona, style, speed). The "reward hacking" analysis, while insightful, is based on small-scale probes (96 rollouts) and may not generalize to all scenarios.
Broader impact
This work has significant implications for the development of more natural and controllable spoken dialogue systems. By providing a benchmark and a training methodology for steerability, it enables the field to move beyond basic turn-taking to more nuanced conversational control. The insights into reward hacking in duplex speech RL are valuable for the broader community working on interactive AI. The separation of content and delivery evaluation is a step towards more holistic assessment of speech models. The paper introduces SteerDuplex, a steerable full-duplex speech model, and SteerBench, a comprehensive benchmark for evaluating spoken steerability. It demonstrates that combining supervised fine-tuning with continuity-aware, two-stage reinforcement learning significantly improves a model's ability to follow instructions regarding tone, persona, and speaking style while maintaining robust interaction behaviors like interruption handling and pause management. The rigorous analysis of reward hacking and the detailed evaluation framework make this a strong contribution to the field of spoken dialogue systems.
Author of this paper? Send a correction.