Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models
Abstract
Large audio-language models (LALMs) are increasingly used for a broader range of audio reasoning tasks. These models typically incorporate audio representations into a large language model (LLM) backbone to enable multimodal reasoning. Recent test-time reinforcement learning (TTRL) methods further improve LLM reasoning capability by leveraging unlabelled test data after pre-training. However, the importance of the perceptual capability of LALMs remains underexplored, particularly how much acoustic evidence is integrated and relied upon during reasoning, and how this contributes to final task performance. This gap limits the development of effective post-training methods like TTRL for audio reasoning. In this work, we first analyse how audio information is integrated and utilised during reasoning process. We quantify layer-wise perceptual reliance and show that stronger acoustic reliance is associated with higher accuracy and a larger performance gain attributable to the audio input. Building on this, we propose Perception-Grounded TTRL (PG-TTRL), which aligns label-free test-time optimisation with perceptually grounded reasoning, encouraging the model to structure its reasoning more strongly on the audio input. Experiments across LALMs and benchmarks show that PG-TTRL consistently improves reasoning performance over both the base models and standard TTRL, showing the value of perceptual-grounding optimisation for test-time audio reasoning.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a rigorous analysis of perceptual reliance in LALMs and proposes PG-TTRL, a novel test-time reinforcement learning method that aligns policy optimization with acoustic grounding, resulting in consistent performance gains over standard TTRL and base models.
Methodology
The paper introduces a novel diagnostic framework for Large Audio-Language Models (LALMs) that quantifies "perceptual reliance" by comparing hidden representations under full audio access versus masked audio attention. This analysis reveals that acoustic evidence integration peaks in intermediate layers and correlates with task accuracy. Building on this, the authors propose PG-TTRL (Perception-Grounded Test-Time Reinforcement Learning), which modifies the Group Relative Policy Optimization (GRPO) advantage function. Instead of relying solely on majority vote correctness, PG-TTRL computes a trajectory-level grounding score based on the log-probability gap between original and masked audio conditions. This score is used to derive a reliability weight that scales the advantage, effectively penalizing updates that rely on language priors rather than acoustic evidence. The method is theoretically sound, addressing a critical flaw in standard TTRL where models may reinforce linguistic shortcuts. The integration of causal intervention (attention masking) into the reward signal is a creative and rigorous approach to grounding multimodal reasoning.
Results & evaluation
Experiments are conducted on two state-of-the-art LALMs (Qwen2.5-Omni-3B and 7B) across two major benchmarks (MMAR and MMAU). The results show consistent improvements over both base models and standard TTRL, with gains up to 4.9 points on MMAU. The analysis of Pass@k metrics is particularly compelling, demonstrating that PG-TTRL preserves diverse correct trajectories better than standard TTRL, which tends to collapse to incorrect majority votes on hard instances. The ablation study on layer-wise changes confirms that the method specifically enhances late-stage reasoning utilization of audio, validating the hypothesis. However, the evaluation is limited to multiple-choice tasks, and the sample size for correlation analysis is modest.
Reproducibility
The paper provides detailed hyperparameters, including LoRA ranks, learning rates, and specific bounds for the reliability weight. The algorithm is clearly defined, and the reproducibility checklist is filled out, indicating that code will be released. The use of standard benchmarks and open-source models (Qwen) enhances reproducibility. However, the specific implementation of the attention masking for the grounding score requires careful handling of the model's internal attention mechanisms, which may be non-trivial to replicate without the provided code.
Limitations
The primary limitation is the restriction to multiple-choice audio reasoning tasks; the method's applicability to open-ended generation or other modalities is not explored. The grounding score relies on teacher-forcing passes, which may not perfectly reflect the autoregressive generation process. Additionally, the method adds computational overhead due to the extra forward passes required for grounding score calculation. The correlation analysis excludes small-sample tasks, which may limit the generalizability of the findings.
Broader impact
This work addresses a fundamental issue in multimodal AI: the tendency of models to ignore perceptual inputs in favor of language priors. By providing a mechanism to explicitly reward perceptual grounding, the paper offers a pathway to more robust and trustworthy multimodal systems. The insights into layer-wise perceptual reliance are valuable for the broader community developing multimodal architectures. The method could be extended to other modalities (vision, video) and tasks, potentially improving the reliability of AI systems in safety-critical applications where accurate perception is crucial. The paper presents a rigorous analysis of perceptual reliance in LALMs and proposes PG-TTRL, a novel test-time reinforcement learning method that aligns policy optimization with acoustic grounding, resulting in consistent performance gains over standard TTRL and base models.
Author of this paper? Send a correction.