ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music Intelligence
Abstract
Omnimodal Notation Processing (ONP) represents a unique frontier for omnimodal AI due to the rigorous, multi-dimensional alignment required across auditory, visual, and symbolic domains. Current research remains fragmented, focusing on isolated transcription tasks that fail to bridge the gap between superficial pattern recognition and the underlying musical logic. This landscape is further complicated by severe notation biases toward Western staff and the inherent unreliability of "LLM-as-a-judge" metrics, which often mask structural reasoning failures with systemic hallucinations. To establish a more rigorous standard, we introduce ONOTE, a multi-format benchmark that utilizes a deterministic pipeline--grounded in canonical pitch projection--to eliminate subjective scoring biases across diverse notation systems. Our evaluation of leading omnimodal models exposes a fundamental disconnect between perceptual accuracy and music-theoretic comprehension, providing a necessary framework for diagnosing reasoning vulnerabilities in complex, rule-constrained domains.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces ONOTE, a comprehensive benchmark for evaluating Omnimodal Notation Processing, which addresses critical gaps in the assessment of music intelligence systems. The methodology and results presented are significant contributions to the field, paving the way for more effective and interpretable models in music AI.
Methodology
The proposed ONOTE benchmark introduces a structured and deterministic evaluation framework for Omnimodal Notation Processing (ONP), addressing the limitations of existing models that often rely on subjective evaluations. The methodology effectively integrates multiple notation systems and tasks, ensuring a comprehensive assessment of model capabilities across auditory, visual, and symbolic domains. The use of canonical pitch projection and sequence alignment to eliminate biases is particularly innovative, allowing for a more rigorous comparison of model performance.
Results & evaluation
The experiments conducted on leading omnimodal models reveal significant insights into their performance across various tasks, including Visual Score Understanding (VSU), Cross-Format Notation Conversion (CNC), Audio-to-Symbolic Transcription (AST), and Symbolic Music Generation (SMG). The results highlight a clear disconnect between perceptual accuracy and music-theoretic comprehension, underscoring the benchmark's effectiveness in diagnosing reasoning vulnerabilities. The dataset construction and evaluation metrics are well-defined, providing a robust foundation for future research.
Reproducibility
The paper provides detailed implementation details and a clear methodology for constructing the ONOTE benchmark, which enhances reproducibility. The availability of the dataset and code on GitHub further supports the reproducibility of the results, allowing other researchers to validate and build upon the work.
Limitations
While the benchmark addresses several critical issues in music notation processing, it may still be limited by the inherent biases present in the datasets used for training and evaluation. Additionally, the focus on specific notation systems may not fully encompass the diversity of global musical representations, potentially limiting the generalizability of the findings.
Broader impact
The ONOTE benchmark has the potential to significantly influence the field of music intelligence by providing a standardized evaluation framework that encourages the development of more robust and interpretable omnimodal systems. Its implications extend beyond academic research, potentially impacting music education, automated composition, and music analysis tools. The paper introduces ONOTE, a comprehensive benchmark for evaluating Omnimodal Notation Processing, which addresses critical gaps in the assessment of music intelligence systems. The methodology and results presented are significant contributions to the field, paving the way for more effective and interpretable models in music AI.
Author of this paper? Send a correction.