AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models
Abstract
Model intelligence and fast response jointly shape the quality of interaction with speech language models, yet remain difficult to achieve together. Explicit chain-of-thought (CoT) improves reasoning and audio understanding, but generating intermediate reasoning tokens delays responses. Describing fine-grained acoustic cues further lengthens CoT and increases latency. Latent reasoning can reduce this overhead, yet existing methods often trail CoT and remain limited by single-path supervision and reasoning budgets that do not adapt to problem difficulty. We introduce AURAL, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency. To provide initial supervision for latent reasoning, we construct AuralReason-683K: 683K bilingual speech utterances (about 1,000 hours) with concise CoT for emotion recognition, empathetic dialogue, and general reasoning. AURAL-RL then explores beyond these traces, rewarding concise reasoning that yields high-quality answers and adapting reasoning effort to each problem. Across two backbones, AURAL-RL achieves performance comparable to CoT-RL, with larger gains over the respective supervised checkpoints on most metrics. Analysis further shows that harder questions elicit more latent reasoning steps. On Qwen2.5-Omni, it reduces time to the first answer token by 11.8x, from 1.22 to 0.10 s, versus 0.05 s for direct answering.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces AURAL, a novel latent reasoning framework for speech language models that uses Gaussian Mixture Models to predict chunks of continuous latent states, achieving CoT-comparable performance with an 11.8x reduction in time-to-first-token latency. This represents a significant step forward in efficient reasoning for multimodal models, offering a practical solution to the latency-intelligence trade-off in interactive speech systems.
Methodology
The paper proposes AURAL, a framework for latent reasoning in Speech Language Models (SLMs). The core methodological contribution is the replacement of autoregressive Chain-of-Thought (CoT) token generation with a Gaussian Mixture Model (GMM) that predicts chunks of continuous latent states. This is a significant architectural shift, moving from discrete sequential decoding to parallel chunked latent prediction. The use of a GMM to model multiple plausible reasoning paths is a clever solution to the mode collapse problem often seen in point-estimation latent reasoning. The introduction of Chunkwise Semantic Alignment (CSA) and Latent Scheduled Sampling (LSS) are well-motivated technical additions to bridge the gap between continuous latent space and discrete language generation. The reinforcement learning component (AURAL-RL) uses a quality-gated conciseness bonus to adapt reasoning depth, which is a logical extension of the SFT phase.
Results & evaluation
The experiments are extensive, covering two backbones (Qwen2.5-Omni, Kimi-Audio) and multiple benchmarks (EchoMind, MMSU, GPQA, etc.). The construction of the AuralReason-683K dataset is a substantial contribution, providing necessary supervision for a task where data is scarce. The results show that AURAL-RL achieves performance comparable to CoT-RL while significantly reducing latency (11.8x speedup in time to first token). The ablation studies effectively isolate the contributions of the GMM, CSA, and LSS components. The analysis of adaptive reasoning depth based on question difficulty is a strong qualitative and quantitative validation of the RL objective.
Reproducibility
The paper provides detailed hyperparameters, training schedules, and architectural specifics in the appendix. The authors state they will release code and data manifests. The use of standard open-source backbones and clearly defined evaluation protocols enhances reproducibility. However, the reliance on specific proprietary models for judging (DeepSeek-V4-Flash, Gemini-3.5-Flash) may introduce slight variability in open-ended task evaluation, though this is standard in current LLM research.
Limitations
The primary limitation is the computational overhead of the GMM sampling and the latent head, which, while faster than CoT, is still slower than direct answering (0.10s vs 0.05s). The method relies heavily on the quality of the initial CoT supervision; if the CoT traces are flawed, the latent targets will be biased. The evaluation of "empathetic dialogue" relies on LLM judges, which can be subjective. The paper does not extensively discuss the failure modes of the GMM, such as when the mixture components might represent spurious paths rather than valid alternatives.
Broader impact
This work has high potential impact on the development of real-time speech assistants. By decoupling reasoning depth from response latency, it enables more intelligent interactions without the user-perceived lag of explicit CoT. The dataset contribution will benefit the broader community working on speech reasoning. The adaptive reasoning approach could be extended to other modalities and tasks where computational efficiency is critical. The paper introduces AURAL, a novel latent reasoning framework for speech language models that uses Gaussian Mixture Models to predict chunks of continuous latent states, achieving CoT-comparable performance with an 11.8x reduction in time-to-first-token latency. This represents a significant step forward in efficient reasoning for multimodal models, offering a practical solution to the latency-intelligence trade-off in interactive speech systems.
Author of this paper? Send a correction.