TinyMU: A Compact Audio-Language Model for Music Understanding
Abstract
Music understanding and reasoning are central challenges in the Music Information Research field, with applications ranging from retrieval and recommendation to music agents and virtual assistants. Recent Large Audio-Language Models (LALMs) have shown remarkable progress in answering music-related questions by following user instructions. However, their massive scale, often billions of parameters, results in expensive training, slow inference, and limited deployability on edge devices. In this work, we present TinyMU, a lightweight (229M) Music-Language Model (MLM) that achieves performance comparable to much larger LALMs while remaining efficient and compact. To train TinyMU, we introduce MusicSkills-3.5M, a carefully curated, music-grounded question-answering dataset with 3.5M samples. Spanning multiple-choice, binary, and open-ended formats, this dataset provides fine-grained supervision across diverse musical concepts. For its architecture, TinyMU leverages MATPAC++, the SOTA self-supervised audio encoder for fine-grained feature extraction. Paired with a lightweight linear projector, it efficiently aligns audio embeddings with the language model. Through extensive evaluation, we show that TinyMU performs strongly in both basic music understanding and complex reasoning. Notably, on the MuChoMusic benchmark, it achieves 82\% of SOTA LALM's performance despite being 35x smaller, highlighting the potential of small MLMs under constrained computational budgets.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents TinyMU, a compact Music-Language Model that achieves strong performance on music understanding and reasoning tasks while being efficient and deployable. The technical contributions, particularly the innovative dataset and the architecture design, mark a significant advancement in the field of music information retrieval and audio-language models.
Methodology
The methodology presented in this paper is robust, focusing on the development of TinyMU, a compact Music-Language Model that leverages a novel dataset, MusicSkills-3.5M, and a state-of-the-art audio encoder, MATPAC++. The authors effectively combine diverse question-answering formats to enhance the model's reasoning and understanding capabilities. The architecture is well-structured, utilizing a lightweight linear projector to align audio and language embeddings, which is a practical approach for compact models. The ablation studies are comprehensive, providing insights into the contributions of different components, which strengthens the validity of the findings.
Results & evaluation
The experiments conducted are thorough, comparing TinyMU against several state-of-the-art models across multiple benchmarks. The results demonstrate that TinyMU achieves competitive performance despite its significantly smaller size, which is a notable achievement in the field. The evaluation metrics used, such as METEOR and BERT-Score, are appropriate for the tasks at hand, and the zero-shot evaluation on independent datasets adds credibility to the results. However, the paper could benefit from more detailed discussions on the statistical significance of the results.
Reproducibility
While the paper mentions that codes and data are available, it lacks specific URLs for the project or demo, which could hinder reproducibility. The methodology is described in sufficient detail, but without access to the actual implementation, it may be challenging for other researchers to replicate the findings fully. Clearer documentation and availability of the code would enhance reproducibility.
Limitations
One limitation of the study is the reliance on the quality and diversity of the MusicSkills-3.5M dataset, which, while comprehensive, may still have biases inherent in the data sources used. Additionally, the model's performance on more complex reasoning tasks may still lag behind larger models, indicating that further improvements are necessary for broader applicability. The paper does not sufficiently address potential ethical considerations in music understanding and generation, which is an important aspect of AI research.
Broader impact
The implications of this research are significant, as it addresses the need for efficient models that can operate in resource-constrained environments, making music understanding technology more accessible. The development of a compact model like TinyMU could enable real-time applications in music recommendation systems, virtual assistants, and educational tools, thus broadening the reach of AI in the music domain. This paper presents TinyMU, a compact Music-Language Model that achieves strong performance on music understanding and reasoning tasks while being efficient and deployable. The technical contributions, particularly the innovative dataset and the architecture design, mark a significant advancement in the field of music information retrieval and audio-language models.
Author of this paper? Send a correction.