Long Listening Thoughts: Eliciting Open Auditory Reasoning with Deliberative Perception and Cognitive Refinement
Abstract
We introduce Long Listening Thoughts (LLT), a framework for constructing post-training datasets that improves audio reasoning by pairing deliberate perceptual grounding with explicit cognitive behaviors. On audio understanding and reasoning benchmarks, large audio-language models (LALMs) are limited primarily by perceptual errors, yet existing post-training datasets rely on audio descriptions of limited diversity. Through a four-stage pipeline, LLT (1) builds multi-aspect dense audio descriptions spanning semantic, acoustic, and temporal information, (2) generates more challenging questions, (3) synthesizes incorrect reasoning traces, and (4) extends short reasoning traces into long ones via a thought-continuation strategy that elicits cognitive behaviors such as verification and backtracking. Using LLT, we synthesize 1.6M reasoning traces and train Qwen2.5-Omni-7B with a two-stage pipeline of supervised fine-tuning (SFT) followed by direct preference optimization (DPO). This improves accuracy on MMAU Test-mini (71.5 to 78.2), MMAR (56.7 to 63.7), and MMSU (60.6 to 66.0).
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a scalable four-stage pipeline for constructing post-training datasets that enhance audio reasoning by combining dense multi-aspect perceptual grounding with explicit cognitive behaviors like verification and backtracking, demonstrating significant improvements in LALMs through a two-stage SFT+DPO training strategy.
Methodology
The paper proposes a four-stage data construction pipeline (LLT) to improve audio reasoning in Large Audio-Language Models (LALMs). The core innovation lies in decoupling perceptual grounding from cognitive refinement. Stage 1 creates dense, multi-aspect audio descriptions (semantic, acoustic via spectrogram-VLM, temporal via specialized detectors) to overcome the limited perceptual diversity of existing datasets. Stage 2 augments question difficulty. Stage 3 synthesizes plausible incorrect reasoning traces using an LLM, rather than relying on model-sampled errors which lack diversity. Stage 4 employs a "thought-continuation" strategy to extend short traces into long ones with explicit cognitive behaviors (verification, backtracking). The training strategy is a two-stage process: SFT on correct and refined traces, followed by DPO using preference pairs that contrast incorrect-to-correct traces. This separation is methodologically sound, as the authors demonstrate that including incorrect traces directly in SFT degrades performance due to the model's inability to distinguish intermediate errors from correct reasoning under standard supervision.
Results & evaluation
The experiments are extensive and well-controlled. The authors validate the necessity of multi-aspect descriptions by showing gains when feeding them to a text-only LLM. They ablate the source of incorrect traces (synthetic vs. model-sampled), showing synthetic traces provide better coverage and distributional separation for DPO. The two-stage training is rigorously compared against single-stage SFT variants. Results show consistent improvements over the Qwen2.5-Omni-7B baseline on MMAU, MMAR, and MMSU. Generalization is tested on different source datasets (AudioMCQ) and different backbones (Qwen2-Audio, Qwen3-Omni), showing the pipeline's robustness. The analysis of cognitive triggers confirms that the model actually engages in error correction (8.7% true corrections) rather than just generating longer text.
Reproducibility
High. The authors release the dataset (1.6M traces), trained models, and the full data construction pipeline. All components (Qwen3-Omni-Captioner, Whisper, pyannote, BEATs, etc.) are open-source. Prompts and implementation details are provided in the appendix. The use of standard benchmarks and clear ablation protocols enhances reproducibility.
Limitations
The pipeline relies heavily on the quality of the initial audio CoT dataset (AF-Think) and the capabilities of the teacher LLMs (Qwen3-Next-80B). The 30-second audio limit restricts applicability to long-form audio, though some transfer is observed. The automated verification of traces may miss subtle logical errors, although human inspection mitigates this. The computational cost of generating 1.6M traces with multiple specialized models is significant.
Broader impact
This work provides a scalable framework for enhancing audio reasoning in LALMs, which is crucial for applications in assistive technology, audio surveillance, and multimodal agents. By explicitly teaching cognitive behaviors like verification and backtracking, it moves beyond simple pattern matching to more robust reasoning. The release of the dataset and pipeline will likely accelerate research in audio post-training. The paper introduces a scalable four-stage pipeline for constructing post-training datasets that enhance audio reasoning by combining dense multi-aspect perceptual grounding with explicit cognitive behaviors like verification and backtracking, demonstrating significant improvements in LALMs through a two-stage SFT+DPO training strategy.
Author of this paper? Send a correction.