ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models
Abstract
End-to-end full-duplex Speech Language Models (SLMs) require precise turn-taking for natural interaction. However, optimizing temporal dynamics via standard raw-token reinforcement learning (RL) degrades semantic quality, causing severe generative collapse and repetition. We propose ASPIRin, an interactivity-optimized RL framework that explicitly decouples when to speak from what to say. Using Action Space Projection, ASPIRin maps the text vocabulary into a coarse-grained binary state (active speech vs. inactive silence). By applying Group Relative Policy Optimization (GRPO) with rule-based rewards, it balances user interruption and response latency. Empirical evaluations show ASPIRin optimizes interactivity across turn-taking, backchanneling, and pause handling. Crucially, isolating timing from token selection preserves semantic coherence and reduces the portion of duplicate n-grams by over 50% compared to standard GRPO, effectively eliminating degenerative repetition.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents ASPIRin, an innovative framework that optimizes interactivity in full-duplex Speech Language Models by decoupling timing from content generation, significantly improving conversational dynamics and semantic coherence. The methodology is well-structured, and the experimental results demonstrate substantial contributions to the field of machine learning and conversational AI.
Methodology
The proposed ASPIRin framework introduces a novel approach to reinforcement learning in full-duplex Speech Language Models by decoupling the timing of speech from the content of speech. This is achieved through Action Space Projection, which simplifies the action space to a binary decision of whether to speak or remain silent. The use of Group Relative Policy Optimization with rule-based rewards to balance responsiveness and interruption risk is innovative and addresses a critical challenge in conversational AI. The methodology is well-structured, with clear definitions and a logical flow from problem identification to solution proposal.
Results & evaluation
The empirical evaluations are robust, utilizing a dedicated benchmark (Full-Duplex-Bench) that assesses various dimensions of interactivity. The results demonstrate significant improvements over baseline models, particularly in reducing generative collapse and enhancing semantic coherence. The experiments are well-documented, and the comparison with multiple baselines strengthens the validity of the findings. However, the paper could benefit from a more detailed discussion on the statistical significance of the results.
Reproducibility
The paper provides sufficient details regarding the training setup, including data collection, model architecture, and hyperparameters. However, the absence of a publicly available code repository or demo limits the reproducibility of the results. Including such resources would enhance the paper's impact and facilitate further research in this area.
Limitations
While the ASPIRin framework shows promising results, it is limited to a binary action space, which may not capture the full complexity of conversational dynamics. Future work is suggested to explore more expressive action spaces. Additionally, the reliance on a specific dataset may limit the generalizability of the findings to other conversational contexts.
Broader impact
The advancements made in this paper have significant implications for the development of more natural and interactive speech systems. By improving the temporal dynamics of full-duplex speech models, this research could enhance user experience in various applications, including virtual assistants, customer service bots, and interactive gaming. The potential for reducing generative repetition also contributes to the overall quality of generated speech. The paper presents ASPIRin, an innovative framework that optimizes interactivity in full-duplex Speech Language Models by decoupling timing from content generation, significantly improving conversational dynamics and semantic coherence. The methodology is well-structured, and the experimental results demonstrate substantial contributions to the field of machine learning and conversational AI.
Author of this paper? Send a correction.