TASTE: Taxonomy-Aware Symbolic Piano Generation from Textual Description
Abstract
Text-conditioned symbolic music generation has attracted growing interest with the development of generative models. Yet two questions remain underexplored: what aspects of music the conditioning text should describe, and how the corresponding data should be collected and trained to improve controllability and text-music alignment. We address these challenges with TASTE, a taxonomy-aware framework for symbolic music generation with a main focus on piano pieces. First, we introduce a taxonomy of musical elements that specifies how a piece can be captured through textual description in a musically grounded way. The taxonomy comprises a set of metadata tags (genre, emotion, section plan, etc.) together with a free-form description of music imagery. This structure decomposes textual input to match the generation goal more coherently and precisely. Second, we build a data pipeline for collecting, annotating, and retrieving public symbolic piano resources, yielding a corpus of 432K MIDI files labeled at multiple levels of granularity. Finally, we design a staged training procedure that leverages this multi-granularity data across two model families: an autoregressive model and a non-autoregressive diffusion model. Both objective and subjective evaluations show that our models achieve superior musical-element control, text-description alignment, and overall music quality. We will release the full pipeline, all annotated datasets, and the model checkpoints.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a taxonomy-aware framework for symbolic piano generation that effectively bridges the gap between user intent and model control. By decomposing musical descriptions into structured tags, section plans, and imagery, and building a robust multi-tier data pipeline to annotate 432K MIDI files, the authors demonstrate significant improvements in attribute agreement, temporal structure, and text-music alignment over existing baselines. The rigorous evaluation, including both objective metrics and human listening studies, validates the effectiveness of the proposed staged training procedure for both autoregressive and diffusion models, establishing a new standard for controllable symbolic music generation.
Methodology
The paper proposes TASTE, a framework that addresses the "description-intention gap" in text-conditioned symbolic music generation. The core methodological contribution is a taxonomy that decomposes musical description into three distinct channels: (1) objective tags (key, tempo, emotion, genre, etc.), (2) section plans (temporal structure), and (3) free-form imagery descriptions. This decomposition allows for targeted annotation and evaluation. The data pipeline is sophisticated, utilizing a multi-tier approach (high-quality human-annotated, curated web-search/LLM-annotated, and large-scale weakly-supervised) to build a corpus of 432K MIDI files. The model architecture integrates these channels into both an Autoregressive (AR) transformer and a Diffusion Transformer (DiT). The AR model uses AdaLN-Zero for tag conditioning and cross-attention for imagery, while the DiT uses similar conditioning mechanisms in a latent space. The staged training procedure, which pretrains on large-scale data with estimated attributes before fine-tuning on high-quality labeled data, is a sound strategy for leveraging weak supervision.
Results & evaluation
The evaluation is comprehensive, covering objective metrics (tag agreement, section boundary F1, FAD, FMD, Audiobox scores) and subjective listening studies. The results show that TASTE models outperform baselines like MuseCoco, text2midi, and MIDI-LLM in both attribute control and text-music alignment. Notably, the DiT model achieves superior temporal structure realization (boundary F1 of 0.868) compared to the AR model (0.517), suggesting that the non-autoregressive approach handles long-range structural dependencies better. The listening study confirms that listeners can match TASTE outputs to their intended imagery descriptions significantly better than chance and even better than reference performances in some cases, indicating strong semantic alignment.
Reproducibility
The paper provides extensive details on the data pipeline, annotation protocols, model architectures, and training configurations. The authors commit to releasing the full pipeline, annotated datasets, and model checkpoints. The use of standard components (Qwen3, CLaMP3, Aria embeddings) and clear descriptions of the weak supervision strategies (probes, retrieval, rewriting) enhance reproducibility. The detailed appendices on probe calibration and confidence gates are particularly helpful for replicating the data construction process.
Limitations
The primary limitation identified is the reliability of labels for "accompaniment" and "performance style," which remain the hardest fields to control due to lower precision in the weak supervision pipeline. The evaluation is focused on piano music, limiting generalizability to other instruments or full orchestral scores. The reliance on LLMs for annotation (web search and audio description) introduces potential biases and inconsistencies, although mitigated by the multi-tier verification process. The AR model's weaker performance in temporal structure realization compared to DiT is a notable weakness for that specific architecture.
Broader impact
This work contributes to the field of controllable music generation by providing a principled framework for describing and controlling musical attributes. The taxonomy and data pipeline can be extended to other musical domains. The release of a large, multi-granularity annotated dataset will benefit the research community by enabling fairer comparisons and more fine-grained control in future models. The integration of symbolic and perceptual conditioning offers a path toward more human-like interaction with music generation systems. The paper introduces a taxonomy-aware framework for symbolic piano generation that effectively bridges the gap between user intent and model control. By decomposing musical descriptions into structured tags, section plans, and imagery, and building a robust multi-tier data pipeline to annotate 432K MIDI files, the authors demonstrate significant improvements in attribute agreement, temporal structure, and text-music alignment over existing baselines. The rigorous evaluation, including both objective metrics and human listening studies, validates the effectiveness of the proposed staged training procedure for both autoregressive and diffusion models, establishing a new standard for controllable symbolic music generation.
Author of this paper? Send a correction.