AudioMap: Cloze-and-Choice Reinforcement Learning for Time-Aware Dense Audio Captioning
Abstract
Time-aware dense audio captioning (TDAC) aims to generate multiple fine-grained attributes (dense) of the audio with precise time boundaries (time-aware). Existing methods struggle to achieve these two goals and mainly rely on supervised fine-tuning, yielding sub-optimal performance. While reinforcement learning (RL) shows promise, applying it to TDAC faces two main challenges: (1) existing rewards are too coarse to supervise multi-event, multi-attribute, and multi-relation descriptions in a fine-grained manner; and (2) temporal supervision is difficult for free-form captions, where flexible event-time expressions make reliable event-time correspondence challenging. To address these challenges, we propose AudioMap, a novel RL-based TDAC framework, which shifts to a unified cloze-and-choice reward paradigm. Specifically, we introduce the Evidence Sufficiency Reward (ESR) with an asymmetric hierarchical scoring mechanism to promote fine-grained accuracy and descriptive richness across diverse acoustic dimensions. Furthermore, we design the Event-Conditioned Temporal Reward (ECTR) to structurally bind timestamps to event semantics via temporal IoU, accompanied by a dual-curriculum learning strategy to facilitate the training process. Finally, to support this task, we construct the first time-aware fine-grained audio captioning dataset, AudioMapCap-44K, which contains 44K carefully annotated captions. Extensive experiments across diverse benchmarks show that AudioMap achieves state-of-the-art (SOTA) performance among open-source models and delivers competitive or superior results relative to proprietary models. Project page and release updates are available at https://github.com/ryysayhi/AudioMap.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
AudioMap introduces a novel RL-based framework for time-aware dense audio captioning, utilizing a cloze-and-choice reward paradigm to enhance fine-grained semantic coverage and temporal grounding, achieving state-of-the-art results among open-source models.
Methodology
The paper proposes AudioMap, an RL-based framework for Time-Aware Dense Audio Captioning (TDAC). The core methodological contribution is a shift from coarse scalar rewards to a "cloze-and-choice" paradigm. Specifically, it introduces the Evidence Sufficiency Reward (ESR), which uses a frozen examiner to generate multiple-choice questions about acoustic details, and the Event-Conditioned Temporal Reward (ECTR), which aligns event descriptions with timestamps using temporal IoU. The approach employs Group Relative Policy Optimization (GRPO) with a dual-curriculum strategy (input duration and reward complexity). The methodology is technically sound and addresses specific pain points in dense captioning (lack of fine-grained supervision and temporal grounding). However, the use of cloze tests for reward modeling is not entirely new in the broader LLM literature (e.g., Omni-Cloze), though its application to dense audio captioning with temporal constraints is a novel adaptation. The reliance on a large frozen examiner (Qwen3.6-27B) for reward computation is a significant computational overhead.
Results & evaluation
The authors construct a new dataset, AudioMapCap-44K, with 44K high-quality captions. Experiments show SOTA performance among open-source models on Omni-Cloze, MMSU, MMAU, and TACOS benchmarks. The results are competitive with proprietary models like Gemini-3.1-Pro. The ablation studies effectively demonstrate the contribution of ESR (semantic coverage) and ECTR (temporal grounding). The user study provides subjective validation. The evaluation is comprehensive, covering semantic, QA-based, and temporal metrics. The comparison with proprietary models is strong.
Reproducibility
The paper provides a GitHub link. The methodology describes the training setup (Qwen2.5-Omni initialization, GRPO details, curriculum stages). However, the reliance on proprietary models for dataset construction (Gemini-3.1-Pro, GPT-4.1) and the specific prompt engineering for the cloze question generation might make exact replication difficult without the specific prompts or access to those models. The code release is a positive sign.
Limitations
The method is computationally expensive due to the need for a large examiner model during RL training. The dataset construction relies heavily on proprietary LLMs, which may introduce biases or limitations in the diversity of the data. The temporal grounding is still dependent on the examiner's ability to extract timestamps, which can be error-prone for concurrent events. The paper does not extensively discuss the failure cases of the temporal reward in complex acoustic scenes.
Broader impact
This work advances the field of audio understanding by enabling more detailed and temporally precise audio descriptions, which is crucial for accessibility, multimedia retrieval, and embodied AI. The dataset AudioMapCap-44K is a valuable resource for the community. The RL-based approach offers a generalizable framework for other dense captioning tasks. AudioMap introduces a novel RL-based framework for time-aware dense audio captioning, utilizing a cloze-and-choice reward paradigm to enhance fine-grained semantic coverage and temporal grounding, achieving state-of-the-art results among open-source models.
Author of this paper? Send a correction.