Audio ML PapersLandmark papers
← All papers
RESEARCH PAPER / 2026-11-16

Think2Listen: Evidence-Centric Learning For Agentic Audio Understanding With LALMS

Jin Wang, Liang Lin, Kaiwen Luo, Weiliu Wang, Yuxuan Li, Zengxin Han, Gan Zhang, Jingtong Wu, Yufei Guo, Kun Wang

Ant International +5 · arXiv

Affiliations: Xi'an University of Electronic Science and Technology, Ant International, Nanyang Technological University, Beijing University of Posts and Telecommunications, University of the Chinese Academy of Sciences, Peking University

Abstract

Recent advances in large audio-language models (LALMs) have broadened audio understanding across speech, sound, and music, with tool-based agents further supporting these tasks by invoking predefined audio tools. However, a persistent gap remains between perceiving acoustic content and using it as evidence for reasoning: recognizing what is audible does not ensure that a model can identify which cues resolve an uncertain decision. To bridge this gap, we propose Think2Listen, an evidence-centric learning framework for agentic audio understanding. Guided by evidence before commitment, Think2Listen couples iterative reasoning with a targeted LALM interaction to actively seek acoustic cues that distinguish competing hypotheses and revise candidate judgments based on the observations returned. It learns what to ask, how to use evidence, and when to stop, using observation-induced changes in candidate-state to inform training credit. Extensive evaluations on two benchmarks spanning seven audio categories demonstrate that Think2Listen achieves (I) enhanced multi-step audio reasoning, with gains in overall accuracy of 5.08%--17.67% over competing audio agents; and (II) improved audio understanding and reasoning across speech, sound, and music, boosting overall accuracy by 2.29%--13.10% over the corresponding baselines. The source code is available at https://github.com/JinWang79/Think2Listen.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

The paper introduces an evidence-centric learning framework for agentic audio understanding that couples iterative reasoning with targeted LALM interactions, using observation-induced candidate-state changes to inform policy training. This approach effectively bridges the gap between acoustic perception and discriminative reasoning, demonstrating significant performance gains over existing audio agents by learning what to ask, how to use evidence, and when to stop.

Methodology

The paper proposes "Think2Listen," a framework that integrates an "Adaptive Audio Evidence Loop" with a novel training signal called "Transition-as-Credit." The methodology addresses the gap between acoustic perception and evidence-based reasoning by allowing an agent to iteratively query a Large Audio-Language Model (LALM) for specific acoustic cues before committing to an answer. The core technical contribution is the "Transition-as-Credit" mechanism, which uses a semantic extractor to reconstruct how observations change the agent's candidate state (eliminating or confirming hypotheses). This process-level feedback is used to calibrate rewards in a Group Relative Policy Optimization (GRPO) style update, distinguishing between useful evidence-seeking and misleading or redundant queries. The approach is theoretically sound, leveraging intermediate state changes to provide denser supervision signals than terminal accuracy alone.

Results & evaluation

The authors evaluate the framework on two benchmarks spanning seven audio categories (speech, sound, music). The results show significant improvements over competing audio agents (5.08%–17.67% accuracy gain) and baselines (2.29%–13.10% gain). The inclusion of ablation studies on the "Transition-as-Credit" component and the analysis of untrained interaction patterns (Figure 2) provides strong empirical support for the necessity of the proposed learning strategy. The experiments effectively demonstrate that learning *what* to ask and *when* to stop is crucial for agentic audio understanding.

Reproducibility

The paper provides a GitHub repository link, which is a positive indicator for reproducibility. The methodology is described with sufficient detail regarding the state formulation, the semantic extractor's role, and the reward calibration formulas. However, specific hyperparameters for the LALM and the actor policy are likely in the appendix (truncated in the provided text), which is standard. The reliance on a specific LALM for the "listener" role may limit immediate reproducibility if that model is not open-source or easily accessible, but the framework itself is clearly defined.

Limitations

The framework relies on the quality of the underlying LALM for generating observations; if the LALM hallucinates acoustic features, the agent's reasoning will be flawed, although the "misleading trajectory" flag attempts to mitigate this. The computational cost of iterative querying and the semantic extraction step for every turn may be high compared to single-shot inference. Additionally, the method is tested on specific benchmarks; generalization to open-ended audio tasks without predefined candidate sets $C$ is not explicitly addressed.

Broader impact

This work contributes to the emerging field of agentic multimodal AI, specifically in the audio domain. By formalizing "evidence-centric" learning, it offers a template for other modalities where perception and reasoning are decoupled. The technique of using state-transition changes for credit assignment is applicable to other tool-using agents. It advances the capability of AI systems to handle complex, multi-step audio reasoning tasks that require active information gathering rather than passive listening. The paper introduces an evidence-centric learning framework for agentic audio understanding that couples iterative reasoning with targeted LALM interactions, using observation-induced candidate-state changes to inform policy training. This approach effectively bridges the gap between acoustic perception and discriminative reasoning, demonstrating significant performance gains over existing audio agents by learning what to ask, how to use evidence, and when to stop.

Author of this paper? Send a correction.