PHALAR: Phasors for Learned Musical Audio Representations
Abstract
Stem retrieval, the task of matching missing stems to a given audio submix, is a key challenge currently limited by models that discard temporal information. We introduce PHALAR, a contrastive framework achieving a relative accuracy increase of up to $\approx 70\%$ over the state-of-the-art while requiring $<50\%$ of the parameters and a 7$\times$ training speedup. By utilizing a Learned Spectral Pooling layer and a complex-valued head, PHALAR enforces pitch-equivariant and phase-equivariant biases. PHALAR establishes new retrieval state-of-the-art across MoisesDB, Slakh, and ChocoChorales, correlating significantly higher with human coherence judgment than semantic baselines. Finally, zero-shot beat tracking and linear chord probing confirm that PHALAR captures robust musical structures beyond the retrieval task.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
PHALAR introduces a novel contrastive framework for musical audio representation that emphasizes phase and pitch equivariance. This work significantly advances the understanding of musical coherence in audio processing, providing a powerful tool for both academic research and practical applications in music technology.
Methodology
The methodology presented in PHALAR is innovative, leveraging complex-valued neural networks (CVNNs) and a Learned Spectral Pooling layer to address the challenge of musical coherence in audio representation. The paper effectively shifts the paradigm from traditional pooling methods that enforce invariance to a framework that emphasizes equivariance, allowing for the preservation of temporal alignment critical for music tasks. The use of phase information as a geometric representation is a significant advancement, as it directly addresses limitations in existing models that fail to capture rhythmic and harmonic relationships adequately.
Results & evaluation
The experimental evaluation is robust, demonstrating PHALAR's superiority over state-of-the-art models in stem retrieval tasks across multiple datasets (MoisesDB, Slakh, and ChocoChorales). The paper includes a comprehensive set of experiments, including human-centered evaluations that correlate model outputs with human perception of coherence. The results indicate a significant improvement in retrieval accuracy and alignment with human judgment, validating the effectiveness of the proposed approach.
Reproducibility
The authors provide a GitHub repository with code and checkpoints, enhancing the reproducibility of their work. They detail the experimental setup, including dataset construction, training parameters, and evaluation metrics, which allows other researchers to replicate their findings. However, the reliance on specific hardware (NVIDIA A100 GPUs) may pose challenges for some researchers in terms of accessibility.
Limitations
The paper acknowledges several limitations, including the model's performance degradation in the presence of non-periodic rhythms and tempo drift, which can affect phase coherence. Additionally, the model's training on predominantly Western popular music may limit its applicability to other musical styles. The potential for rigid enforcement of rhythmic standards in automated systems is also noted as a concern.
Broader impact
PHALAR has significant implications for music information retrieval and audio processing, offering a novel approach that could enhance various applications, including automated music production and generative audio models. The framework's emphasis on phase-sensitive data representation could extend to other fields such as radar systems and medical imaging, where preserving complex signal integrity is crucial. PHALAR introduces a novel contrastive framework for musical audio representation that emphasizes phase and pitch equivariance. This work significantly advances the understanding of musical coherence in audio processing, providing a powerful tool for both academic research and practical applications in music technology.
Author of this paper? Send a correction.