Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models
Abstract
Recent text-to-audio models generate high-quality audio, but often fail to follow instructions involving multiple sound events and temporal order. This gap arises because existing evaluation and training signals mainly emphasize global similarity or perceptual quality, with limited supervision on instruction-level correctness. We propose an instruction-level framework that uses audio-aware large language models (ALLMs) as fine-grained judges to verify target event presence and temporal relations in generated audio. After validating ALLM judgments on benchmarks and through human verification, we use their feedback to construct preference pairs for direct preference optimization. We further introduce S3Bench, a narrative benchmark for evaluating multi-event temporal instruction following. Experiments show that our method improves event completeness, temporal ordering, and joint instruction-following accuracy across existing benchmarks and S3Bench, while maintaining audio quality.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a significant advancement in Text-to-Audio generation by introducing a novel preference optimization framework that leverages Audio-Aware LLMs for fine-grained instruction verification, effectively bridging the gap between global audio quality and precise temporal event control, while contributing a valuable new benchmark to the community.
Methodology
The paper proposes a novel framework, ALLM-Judged Preference Optimization (AJPO), to address the lack of fine-grained instruction following in Text-to-Audio (TTA) generation. The core innovation lies in leveraging Audio-Aware Large Language Models (ALLMs) as structured judges to verify sound event existence and temporal ordering, rather than relying on global similarity metrics like CLAPScore. These structured judgments are used to construct preference pairs for Direct Preference Optimization (DPO). The methodology is sound and addresses a specific, well-identified gap in the field: the misalignment between global perceptual quality and fine-grained instruction adherence. The use of ALLMs as reward models/judges is a timely approach, capitalizing on the emergent reasoning capabilities of recent multimodal models. However, the novelty is somewhat incremental as the concept of using LLMs/ALLMs for preference optimization is becoming more common across modalities; the specific application to temporal event ordering in audio is the key differentiator.
Results & evaluation
The experimental evaluation is comprehensive and robust. The authors first validate the capability of ALLMs (Qwen2.5-Omni, Audio Flamingo) as judges on existing benchmarks and synthetic datasets, showing strong correlation with human judgments. They then introduce S3Bench, a new benchmark for narrative multi-event audio, which adds significant value to the community. The ablation studies effectively demonstrate the superiority of fine-grained ALLM feedback over CLAP-based ranking and caption-based matching. The comparison with existing preference datasets (Audio-Alpaca, Baton) highlights the benefits of dynamic, instruction-aligned preference construction. The results show consistent improvements in event completeness and temporal ordering without degrading audio quality metrics (FAD, IS, CLAPScore). The inclusion of online DPO experiments further strengthens the technical contribution by showing the importance of dynamic preference reconstruction.
Reproducibility
The paper provides sufficient detail for reproduction, including model architectures (TangoFlux-base), training hyperparameters (learning rate, batch size, warm-up steps), and the construction of the S3Bench dataset. The use of open-source models (Qwen2.5-Omni, Audio Flamingo) and standard TTA baselines enhances reproducibility. The project page link suggests availability of qualitative examples and potentially code, which would further aid reproducibility.
Limitations
The approach relies heavily on the accuracy of the ALLM judge. While validated, ALLMs can still hallucinate or make errors, particularly in complex, noisy audio scenarios (as noted in the evaluation of AudioCaps-test vs. AudioTime). The performance of the TTA model is thus bounded by the judge's reliability. Additionally, the synthetic MultiEvent-Temporal benchmarks, while useful for controlled evaluation, may not fully capture the complexity of real-world audio scenes with overlapping sounds and background noise, although S3Bench attempts to mitigate this. The computational cost of running ALLM judgments for preference construction, especially in online DPO, is non-trivial.
Broader impact
This work significantly advances the controllability of TTA systems, making them more suitable for complex applications like film scoring, game audio design, and assistive technologies where precise instruction following is critical. The introduction of S3Bench provides a standardized benchmark for future research in this area. The methodological framework of using multimodal LLMs for fine-grained preference optimization could be generalized to other generative tasks. The paper presents a significant advancement in Text-to-Audio generation by introducing a novel preference optimization framework that leverages Audio-Aware LLMs for fine-grained instruction verification, effectively bridging the gap between global audio quality and precise temporal event control, while contributing a valuable new benchmark to the community.
Author of this paper? Send a correction.