Closing the Modality Reasoning Gap for Speech Large Language Models
Abstract
Although speech large language models have achieved notable progress, a substantial modality reasoning gap remains: their reasoning performance on speech inputs is markedly weaker than on text. This gap could be associated with representational drift across Transformer layers and behavior deviations in long-chain reasoning. To address this issue, we introduce TARS, a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design. The framework employs two dense and complementary signals: representation alignment, which measures layer-wise hidden-state similarity between speech- and text-conditioned trajectories, and behavior alignment, which evaluates semantic consistency between generated outputs and reference text completions. Experiments on challenging reasoning benchmarks, including MMSU and OBQA, show that our approach significantly narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents TARS, a reinforcement-learning framework that effectively narrows the modality reasoning gap in speech LLMs. This contribution is significant as it enhances the reasoning capabilities of speech models, potentially leading to more advanced applications in spoken language processing.
Methodology
The paper introduces TARS, a novel reinforcement-learning framework designed to address the modality reasoning gap in speech large language models (LLMs). The methodology is well-structured, employing an asymmetric reward design that incorporates both representation alignment and behavior alignment. This dual approach effectively mitigates representational drift and enhances semantic consistency in reasoning tasks. The use of on-policy exploration to reduce exposure bias is a significant innovation, allowing the model to learn from its own reasoning trajectories rather than relying solely on static targets. The framework's reliance on dense rewards is a notable advancement over traditional sparse reward systems in reinforcement learning.
Results & evaluation
The experiments are comprehensive, utilizing challenging reasoning benchmarks (MMSU and OBQA) to validate the effectiveness of the proposed method. The results demonstrate a significant narrowing of the modality reasoning gap, achieving state-of-the-art performance among 7B-scale speech LLMs. The paper provides a clear comparison against various baselines, showcasing the advantages of the proposed approach. However, the evaluation could benefit from additional metrics or broader datasets to further substantiate the claims of superiority.
Reproducibility
The paper mentions the release of datasets and code to facilitate reproducibility, which is a positive aspect. However, specific details regarding the implementation, hyperparameters, and training protocols are scattered throughout the text, which may hinder ease of reproduction for other researchers. A dedicated section summarizing these details would enhance clarity.
Limitations
The paper acknowledges several limitations, including the focus on single-turn reasoning and the potential inadequacy of the proposed alignment rewards in capturing paralinguistic cues. Additionally, the evaluation is limited to 7B-scale models, leaving questions about the framework's scalability to larger architectures. These limitations suggest areas for future research and improvement.
Broader impact
The proposed framework has significant implications for the development of more robust and capable speech LLMs, which could enhance applications in automatic speech recognition, spoken dialogue systems, and other areas of natural language processing. By addressing the modality reasoning gap, the research contributes to making speech-based AI systems more effective and reliable. The paper presents TARS, a reinforcement-learning framework that effectively narrows the modality reasoning gap in speech LLMs. This contribution is significant as it enhances the reasoning capabilities of speech models, potentially leading to more advanced applications in spoken language processing.
Author of this paper? Send a correction.