Masked Contrastive Pre-Training Improves Music Audio Key Detection
Abstract
Self-supervised music foundation models underperform on key detection, which requires pitch-sensitive representations. In this work, we present the first systematic study showing that the design of self-supervised pretraining directly impacts pitch sensitivity, and demonstrate that masked contrastive embeddings uniquely enable state-of-the-art (SOTA) performance in key detection in the supervised setting. First, we discover that linear evaluation after masking-based contrastive pretraining on Mel spectrograms leads to competitive performance on music key detection out of the box. This leads us to train shallow but wide multi-layer perceptrons (MLPs) on features extracted from our base model, leading to SOTA performance without the need for sophisticated data augmentation policies. We further analyze robustness and show empirically that the learned representations naturally encode common augmentations. Our study establishes self-supervised pretraining as an effective approach for pitch-sensitive MIR tasks and provides insights for designing and probing music foundation models.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of KeyMyna, a novel approach to music key detection that leverages masked contrastive pretraining to achieve state-of-the-art performance without complex data augmentation. This work significantly advances the field of music information retrieval by demonstrating the potential of self-supervised learning techniques in capturing pitch-sensitive representations, thereby addressing a critical challenge in the domain.
Methodology
The paper introduces KeyMyna, a systematic study of self-supervised pretraining for music key detection using masked contrastive learning. The methodology is well-structured, leveraging a pre-trained model (Myna-Vertical) and shallow multi-layer perceptrons (MLPs) for key detection. The authors effectively demonstrate the advantages of their approach over traditional methods and other deep learning models, particularly in terms of pitch sensitivity and robustness to augmentations. The use of a simple contrastive learning framework with token masking is innovative and addresses the challenges of limited labeled datasets in music key detection.
Results & evaluation
The experiments are thorough, utilizing two widely recognized datasets (GiantSteps and McGill Billboard) for evaluation. The results show that KeyMyna outperforms existing methods despite using less data and simpler architectures. The paper provides a comprehensive comparison with prior work, demonstrating the effectiveness of their approach through various metrics. However, the paper could benefit from more extensive ablation studies to further validate the impact of individual components of their methodology.
Reproducibility
The authors provide a GitHub repository with code and models, which is a positive aspect for reproducibility. However, detailed hyperparameter settings and training configurations are presented, but the absence of a complete training script or environment setup instructions may hinder full reproducibility for some researchers.
Limitations
The paper acknowledges limitations, such as the inability of KeyMyna to track key modulations within songs, which could affect performance in certain musical genres. Additionally, the focus on major and minor keys limits the model's applicability to more complex musical structures. Future work is suggested to address these limitations, including the exploration of moving averages for key modulation detection.
Broader impact
The findings of this research have significant implications for music information retrieval (MIR) applications, including playlist generation and music similarity search. By improving key detection through self-supervised learning, the work contributes to the development of more robust and efficient MIR systems. The insights gained from this study could also inform future research in music analysis and representation learning. The main contribution of this paper is the introduction of KeyMyna, a novel approach to music key detection that leverages masked contrastive pretraining to achieve state-of-the-art performance without complex data augmentation. This work significantly advances the field of music information retrieval by demonstrating the potential of self-supervised learning techniques in capturing pitch-sensitive representations, thereby addressing a critical challenge in the domain.
Author of this paper? Send a correction.