The ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge
Abstract
This paper summarizes the ICASSP 2026 Automatic Song Aesthetics Evaluation (ASAE) Challenge, which focuses on predicting the subjective aesthetic scores of AI-generated songs. The challenge consists of two tracks: Track 1 targets the prediction of the overall musicality score, while Track 2 focuses on predicting five fine-grained aesthetic scores. The challenge attracted strong interest from the research community and received numerous submissions from both academia and industry. Top-performing systems significantly surpassed the official baseline, demonstrating substantial progress in aligning objective metrics with human aesthetic preferences. The outcomes establish a standardized benchmark and advance human-aligned evaluation methodologies for modern music generation systems.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper effectively establishes a benchmark for evaluating the aesthetics of AI-generated music, advancing methodologies in the field and fostering collaboration among researchers. The challenge's design and results highlight the importance of aligning objective metrics with human preferences, paving the way for future advancements in music generation technologies.
Methodology
The paper presents a well-structured challenge that addresses a significant gap in the evaluation of AI-generated music aesthetics. The use of two distinct tracks for overall musicality and fine-grained aesthetic dimensions is innovative, allowing for comprehensive evaluation. The hierarchical evaluation metric proposed is a strong methodological contribution, as it combines multiple correlation metrics to assess model performance effectively. The challenge's design encourages participants to explore diverse modeling strategies, which is crucial for advancing the field.
Results & evaluation
The experimental setup is robust, with a clear delineation of training and test datasets, including both seen and unseen systems. The performance metrics used provide a thorough assessment of model capabilities. The results indicate that top-performing systems have made significant strides in aligning with human aesthetic preferences, showcasing the effectiveness of various modeling techniques. However, the paper could benefit from more detailed statistical analysis of the results to better understand the performance variations among different models.
Reproducibility
The paper provides adequate information regarding the datasets and evaluation metrics, which is essential for reproducibility. However, it lacks detailed descriptions of the individual models used by the participants, which could hinder full reproducibility of results. Including links to the submitted models or detailed descriptions of their architectures would enhance reproducibility.
Limitations
One notable limitation is the modest TTA scores across teams, suggesting that while models can rank songs well, they struggle with fine-grained discrimination, particularly at the high-quality end. Additionally, the challenge's reliance on subjective aesthetic scores may introduce variability based on individual preferences, which could affect the generalizability of the findings.
Broader impact
The challenge has the potential to significantly influence the field of music generation by providing a standardized benchmark for evaluating AI-generated music. It encourages collaboration between academia and industry, fostering innovation in music technology. The outcomes could lead to improved music generation systems that better align with human aesthetic preferences, impacting both the creative industries and consumer experiences. The paper effectively establishes a benchmark for evaluating the aesthetics of AI-generated music, advancing methodologies in the field and fostering collaboration among researchers. The challenge's design and results highlight the importance of aligning objective metrics with human preferences, paving the way for future advancements in music generation technologies.
Author of this paper? Send a correction.