Count The Notes: Histogram-Based Supervision for Automatic Music Transcription
Abstract
Automatic Music Transcription (AMT) converts audio recordings into symbolic musical representations. Training deep neural networks (DNNs) for AMT typically requires strongly aligned training pairs with precise frame-level annotations. Since creating such datasets is costly and impractical for many musical contexts, weakly aligned approaches using segment-level annotations have gained traction. However, existing methods often rely on Dynamic Time Warping (DTW) or soft alignment loss functions, both of which still require local semantic correspondences, making them error-prone and computationally expensive. In this article, we introduce CountEM, a novel AMT framework that eliminates the need for explicit local alignment by leveraging note event histograms as supervision, enabling lighter computations and greater flexibility. Using an Expectation-Maximization (EM) approach, CountEM iteratively refines predictions based solely on note occurrence counts, significantly reducing annotation efforts while maintaining high transcription accuracy. Experiments on piano, guitar, and multi-instrument datasets demonstrate that CountEM matches or surpasses existing weakly supervised methods, improving AMT's robustness, scalability, and efficiency. Our project page is available at https://yoni-yaffe.github.io/count-the-notes.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
CountEM introduces a novel framework for automatic music transcription that utilizes histogram-based supervision, significantly simplifying the annotation process while maintaining high transcription accuracy. The technical contributions and methodology present a meaningful advancement in the field of music information retrieval, with potential applications across various domains.
Methodology
The methodology presented in CountEM is innovative, leveraging note event histograms for supervision in automatic music transcription (AMT) without requiring explicit local alignment. The use of an Expectation-Maximization (EM) approach to iteratively refine predictions based on histogram counts is a significant departure from traditional methods that rely on Dynamic Time Warping (DTW) and local alignment techniques. The peak-picking mechanism for estimating note onsets is straightforward yet effective, allowing for greater flexibility and reduced computational overhead. The paper effectively communicates the theoretical underpinnings of the approach, making it accessible for further exploration and adaptation.
Results & evaluation
The experiments conducted across various datasets (MAESTRO, GuitarSet, and MusicNet) demonstrate the robustness and generalizability of CountEM. The results indicate that the proposed method matches or surpasses existing weakly supervised methods, showcasing improved transcription accuracy even with large time windows. The evaluation metrics used, including note-level precision, recall, and F-score, are appropriate for the task and provide a comprehensive assessment of the model's performance. However, the paper could benefit from more detailed comparisons with a broader range of existing methods to contextualize its contributions further.
Reproducibility
The paper provides sufficient implementation details, including the architecture used, training procedures, and hyperparameters, which facilitate reproducibility. The use of PyTorch and the mention of specific hardware (NVIDIA GeForce RTX 3090 GPUs) enhances the clarity of the experimental setup. However, the absence of a public code repository may hinder full reproducibility for some researchers.
Limitations
One limitation of the proposed method is its reliance on the quality of the histograms derived from potentially noisy or imprecise musical scores, which could affect transcription accuracy in real-world applications. Additionally, while the method shows promise across various datasets, its performance in more complex polyphonic scenarios or with less structured musical data remains to be fully explored.
Broader impact
The implications of CountEM extend beyond music transcription, potentially influencing related fields such as instrument recognition, rhythm analysis, and even lyrics transcription. By reducing the annotation burden and improving the scalability of AMT systems, this work could democratize access to music transcription technologies, benefiting educators, musicians, and researchers alike. CountEM introduces a novel framework for automatic music transcription that utilizes histogram-based supervision, significantly simplifying the annotation process while maintaining high transcription accuracy. The technical contributions and methodology present a meaningful advancement in the field of music information retrieval, with potential applications across various domains.
Author of this paper? Send a correction.