AllMusicCaps: Album Reviews as Complementary Supervision for Music CLAP
Abstract
Recent open text-audio contrastive models (CLAPs) are typically trained with LLM-generated captions derived from tag datasets or web search results, which tend to be accurate but expressively narrow. As a complementary source, we explore human-written album reviews, specifically expert reviews from AllMusic: they exist at scale and carry narrative cues, evaluative adjectives, and scene framing that other sources lack. Since raw reviews are too noisy for direct use as captions, we first build a caption corpus with 24,5346 samples via an LLM preprocessing pipeline that identifies descriptive musical quotes and rewrites them into training-ready captions. We find that album review supervision yields the largest retrieval gains on a human-written caption benchmark (Song Describer), particularly for complex queries that other existing caption datasets leave uncovered. In addition, we revisit the training recipe and show that SigReg regularization, which encourages an isotropic Gaussian distribution in the embedding space, improves MLP probing across classification tasks, as well as text-to-music retrieval. The resulting model outperforms open CLAP-style baselines on text-to-music retrieval, zero-shot classification, and most MLP probing tasks. We release the review-derived caption dataset and model weights to support future research.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a valuable contribution to music-text alignment by introducing a dataset derived from expert album reviews and demonstrating its efficacy in improving retrieval of complex musical descriptions. The integration of SigReg regularization and detailed ablation studies on training objectives adds technical depth, making it a solid contribution to the field of multimodal audio research.
Methodology
The paper proposes a novel data-centric approach to training text-audio contrastive models (CLAPs) by leveraging expert-written album reviews from AllMusic. The core methodological contribution is an LLM-based preprocessing pipeline that extracts descriptive musical quotes from reviews and rewrites them into training-ready captions. This addresses a known gap in existing datasets (like MusicCaps) which rely on tag-to-text generation and lack narrative or evaluative depth. Additionally, the authors conduct an ablation study on training objectives, specifically investigating the integration of SigReg regularization (from LeJEPA) with the standard InfoNCE loss, and layer selection strategies for the audio encoder. The approach is sound and addresses a specific limitation in current multimodal music representation learning.
Results & evaluation
The evaluation is comprehensive, covering text-to-music retrieval (MusicCaps, Song Describer), zero-shot classification (GTZAN, FMA-Small), audio similarity (DimSim), and MLP probing (MTT, MTG-Jamendo). The results demonstrate that the proposed model outperforms open baselines (Laion-CLAP, TTMR++, CLaMP3) on most metrics, particularly on the Song Describer benchmark which contains complex, human-written captions. The analysis of query complexity (lexical-slice analysis) provides strong evidence that the review-derived data specifically benefits the retrieval of complex, narrative-driven queries. The inclusion of SigReg regularization shows consistent improvements in probing tasks, validating the theoretical benefit of isotropic embeddings for discriminative tasks.
Reproducibility
The authors release the dataset, model weights, and code. The paper provides detailed descriptions of the LLM pipelines, hyperparameters, and training setups. The use of standard benchmarks ensures that results can be compared with future work. The reliance on LLMs for data generation introduces some stochasticity, but the pipeline is described in sufficient detail for replication.
Limitations
The paper acknowledges that review-derived captions alone are insufficient and must be combined with other corpora. The reliance on LLMs for caption generation introduces potential biases or hallucinations, although the authors argue the risk is low based on manual inspection. The evaluation is primarily on English-language data, limiting generalizability. The "AllMusic" dataset is proprietary in its raw form, though the derived captions are released. The venue date "ISMIR 2026" is likely a typo for 2024 or 2025, but does not affect the technical assessment.
Broader impact
This work contributes to the open-source ecosystem for music information retrieval by providing a new dataset and model that capture more nuanced musical descriptions. It encourages the use of diverse textual sources for multimodal training. The release of weights and code supports further research in audio-language alignment. The paper presents a valuable contribution to music-text alignment by introducing a dataset derived from expert album reviews and demonstrating its efficacy in improving retrieval of complex musical descriptions. The integration of SigReg regularization and detailed ablation studies on training objectives adds technical depth, making it a solid contribution to the field of multimodal audio research.
Author of this paper? Send a correction.