MeloBottleneck: Self-Supervised Melody Skeleton Extraction with a Latent Subsequence Bottleneck
Abstract
Melody skeleton extraction aims to derive a shorter melody that preserves structural notes while removing ornaments. Prior methods rely on hand-crafted reduction rules or note-wise salience classifiers trained with heuristically or procedurally generated pseudo-labels. Such supervision can inherit generator bias and does not explicitly optimize a coherent reduced melody. We introduce MeloBottleneck, a self-supervised framework that represents a skeleton as a length-controlled, order-preserving latent subsequence. A hard-bottleneck extractor selects note events, a rhythmic-closure operator produces a self-consistent skeleton, and a re-ornamentation decoder reconstructs the input melody. Training combines reconstruction, a frozen autoregressive melody prior, ornament-invariant consistency across procedurally ornamented views, and ornament exclusion. We evaluate three regimes: synthetic out-of-distribution ornament-to-skeleton, TAVERN variation-to-theme, and Jiugong ornamented-to-gongche. A matched pseudo-label classifier excels on the synthetic benchmark, while MeloBottleneck transfers better, achieving competitive selection quality on TAVERN and Jiugong. Skeletonized melodies also improve BM25-based fragment retrieval, boosting Recall@K and MRR while reducing query time. Overall, the results suggest that learning skeletons as latent subsequences yields more robust transfer than pseudo-label imitation.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
MeloBottleneck introduces a novel self-supervised framework for melody skeleton extraction that models skeletons as latent subsequences, demonstrating superior cross-domain transfer and practical utility in music retrieval compared to prior pseudo-label or heuristic methods.
Methodology
The paper proposes MeloBottleneck, a self-supervised framework for melody skeleton extraction. The core innovation lies in formulating the skeleton not as a set of independent note-wise labels, but as a length-controlled, order-preserving latent subsequence. This is a significant methodological shift from prior work that relies on heuristic rules or pseudo-label classifiers. The architecture employs a Transformer encoder to select top-K notes, followed by a deterministic "rhythmic closure" operator that converts the sparse subsequence into a standalone melody by absorbing deleted time into preceding notes. Training is multi-objective: it includes reconstruction loss (re-ornamentation), a frozen autoregressive melody prior for musicality, and ornament-invariant consistency losses to ensure stability under procedural ornamentation. The approach effectively decouples note selection from rhythmic coherence, addressing a key limitation in previous discrete latent compression methods for music.
Results & evaluation
The evaluation is comprehensive, covering three distinct regimes: synthetic out-of-distribution (OOD) ornament-to-skeleton, cross-domain variation-to-theme (TAVERN), and in-domain ornamented-to-gongche (Jiugong). The results demonstrate a clear trade-off: while a pseudo-label classifier excels on synthetic data where it directly imitates the generator, MeloBottleneck shows superior transferability to zero-shot and cross-domain settings. This supports the central claim that learning structural representations is more robust than imitation. Additionally, the paper provides a practical downstream application, showing that skeletonized melodies improve BM25-based fragment retrieval by reducing noise and query time. The ablation studies rigorously validate the contribution of each loss component, particularly the ornament exclusion and consistency losses.
Reproducibility
The paper provides detailed implementation specifics, including the SimpleMono event representation, hyperparameters for the Transformer backbone (6 encoder/3 decoder layers), and specific loss coefficients. The authors also provide a link to supplementary materials including code and a demo web page, which significantly enhances reproducibility. The use of standard symbolic music representations and clear algorithmic descriptions of the rhythmic closure and selection mechanisms makes the method straightforward to implement.
Limitations
The authors acknowledge several limitations. First, the method relies on a procedural ornamenter for training views; if this ornamenter does not match the target style, the invariance signal may be suboptimal. Second, the evaluation of selection quality uses an oracle compression ratio, meaning free-length prediction is only indirectly tested. Third, the rhythmic closure operator is simple (absorbing time into the preceding note), which may not preserve musical phrasing as well as more complex closure strategies. Finally, the focus is strictly on monophonic melodies, limiting immediate applicability to polyphonic music without significant extension.
Broader impact
This work has significant implications for symbolic music analysis, retrieval, and compression. By providing a robust method to extract structural skeletons, it facilitates better theme tracing, variation recognition, and efficient indexing of large music corpora. The self-supervised nature of the approach reduces reliance on expensive human annotations, making it scalable to diverse musical traditions. The improved retrieval performance suggests practical utility in digital library systems and music information retrieval tools. MeloBottleneck introduces a novel self-supervised framework for melody skeleton extraction that models skeletons as latent subsequences, demonstrating superior cross-domain transfer and practical utility in music retrieval compared to prior pseudo-label or heuristic methods.
Author of this paper? Send a correction.