Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning
Abstract
Recent full-duplex spoken dialogue models have demonstrated compelling progress toward human-like interaction, enabling agents to respond with low latency, produce backchannels, and handle user barge-ins. Yet these improvements in conversational dynamics often come with weaker reasoning and instruction-following abilities, revealing a potential tension between interactive dynamics and intelligence capability. In this paper, we argue that such an intelligence--dynamics trade-off is not fundamental: conversational dynamics can instead be learned as a separate real-time decision policy from human dialogue data. To this end, we propose DuplexPO, a reinforcement learning (RL) framework that decouples when to speak from what to say. It preserves the semantic response capability of an instruction-tuned assistant, while optimizing its temporal interaction behavior over selected high-impact windows from long human conversations. To quantitatively optimize these dynamics, we formulate the Factorized Conversational Dynamics Reward (FCDR) to enable fine-grained temporal credit assignment for turn initiation, backchanneling, yielding, and regularized participation. The policy is then optimized with a GRPO-style objective. Experiments show that DuplexPO substantially improves full-duplex behaviors, including timely backchannels, smooth turn-taking, and barge-in handling, while maintaining strong reasoning and instruction-following performance. Moreover, improvements in dynamics-oriented metrics are reflected in better user experience, suggesting that optimizing conversational timing as a standalone objective can promote more natural full-duplex interaction.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a significant advancement in full-duplex spoken dialogue systems by introducing a novel RL framework that effectively decouples conversational timing from semantic generation, leading to more natural interactions without compromising reasoning capabilities.
Methodology
The paper proposes DuplexPO, a reinforcement learning framework designed to decouple conversational dynamics (when to speak) from semantic content (what to say) in full-duplex spoken dialogue models. The core methodological innovation lies in the Factorized Conversational Dynamics Reward (FCDR), which provides fine-grained temporal credit assignment for specific interaction events like turn initiation, backchanneling, and yielding. By restricting policy optimization to "dynamics-critical windows" sampled from long human conversations and using a GRPO-style objective, the method aims to improve real-time interaction behaviors without degrading the model's instruction-following capabilities. This addresses a specific and challenging problem in end-to-end spoken language models: the tension between low-latency responsiveness and reasoning quality. The approach is technically sound, leveraging established RL techniques (GRPO) in a novel context (temporal windowing for audio dialogue).
Results & evaluation
The evaluation is comprehensive, covering both conversational dynamics metrics (onset MAE, yield rate, voiced interrupt rate) on standard datasets (Fisher, Seamless, Full-Duplex-Bench) and model intelligence benchmarks (QA, instruction following). The results demonstrate that DuplexPO significantly improves dynamic metrics compared to SFT baselines while maintaining or slightly improving intelligence metrics. The inclusion of an LLM-as-a-Judge evaluation for naturalness adds qualitative depth. The ablation studies on window size, reward design, and optimization objectives (GRPO vs. DPO) provide strong empirical support for the design choices. The comparison with commercial models and other open-source full-duplex models places the work in a competitive context.
Reproducibility
The paper provides detailed implementation details, including model architecture (Qwen2.5-7B backbone, Parakeet encoder), training stages, and hyperparameters for the RL phase. The data sources (Fisher, Seamless) are public, and the reconstruction process is described. The inclusion of a demo URL suggests the authors are committed to transparency. However, the exact code for the window sampling and FCDR reward calculation is not explicitly linked in the text (though a GitHub link might be implied by the demo page, it is not in the provided text). The detailed appendix sections on data generation and training pipelines enhance reproducibility.
Limitations
The authors acknowledge that FCDR may miss subtle pragmatic factors like user intent and cultural timing preferences. The reliance on human timing annotations, which are not unique ground truths, is a limitation. Furthermore, restricting optimization to local windows might miss long-range dialogue coherence effects. The evaluation is primarily in English, and the method's generalizability to other languages or highly asynchronous interaction styles is not fully explored.
Broader impact
This work contributes to the development of more natural and responsive voice assistants, which has significant implications for human-computer interaction, accessibility, and AI adoption. The potential for misuse in deceptive practices (e.g., vishing) is noted, highlighting the need for responsible deployment and safeguards. The decoupling of dynamics and semantics offers a generalizable paradigm for optimizing multi-modal agents where timing and content are distinct concerns. This paper presents a significant advancement in full-duplex spoken dialogue systems by introducing a novel RL framework that effectively decouples conversational timing from semantic generation, leading to more natural interactions without compromising reasoning capabilities.
Author of this paper? Send a correction.