Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Abstract
Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of Audio-DeepThinker, a novel framework that enables high-quality chain-of-thought reasoning to emerge in large audio-language models through reinforcement learning, significantly advancing the field of audio reasoning. The combination of innovative methodologies and strong experimental results positions this work as a meaningful contribution to the machine learning community, particularly in the audio domain.
Methodology
The proposed methodology of Audio-DeepThinker is innovative, leveraging a hybrid reasoning similarity reward and a progressive two-stage curriculum to enhance reasoning capabilities in audio-language models. The integration of reinforcement learning without prior supervised fine-tuning is a significant advancement, allowing the model to develop reasoning skills through exploration. The detailed data construction pipeline and the dual reward system provide a structured approach to improving model performance while ensuring acoustic grounding in reasoning chains.
Results & evaluation
The experiments conducted on multiple benchmarks (MMAR, MMAU, and MMSU) demonstrate the effectiveness of Audio-DeepThinker, achieving state-of-the-art results. The comprehensive evaluation metrics, including accuracy and Rubrics scores, validate the model's performance across various audio reasoning tasks. The ablation studies further substantiate the contributions of the hybrid reward and the two-stage training approach.
Reproducibility
While the paper provides a thorough description of the methodology and experimental setup, including hyperparameters and training details, it lacks direct links to code or datasets, which may hinder reproducibility. The absence of a project URL or demo limits the ability for others to replicate the results.
Limitations
One limitation is the reliance on automated data construction, which may introduce biases or inaccuracies in the generated reasoning chains. Additionally, the model's performance on boundary cases may still be constrained by the inherent challenges of audio reasoning, and further exploration of diverse audio contexts is needed to fully assess its robustness.
Broader impact
The advancements presented in Audio-DeepThinker have significant implications for audio understanding and reasoning tasks, potentially enhancing applications in accessibility, education, and interactive audio systems. The framework could pave the way for more sophisticated audio-language models capable of nuanced reasoning, thereby improving user interactions with audio content. The main contribution of this paper is the introduction of Audio-DeepThinker, a novel framework that enables high-quality chain-of-thought reasoning to emerge in large audio-language models through reinforcement learning, significantly advancing the field of audio reasoning. The combination of innovative methodologies and strong experimental results positions this work as a meaningful contribution to the machine learning community, particularly in the audio domain.
Author of this paper? Send a correction.