Depth-Structured Music Recurrence: Budgeted Recurrent Attention for Full-Piece Symbolic Music Modeling
Abstract
Long-context modeling is essential for symbolic music generation, since motif repetition and developmental variation can span thousands of musical events. However, practical composition and performance workflows frequently rely on resource-limited devices (e.g., electronic instruments and portable computers), making heavy memory and attention computation difficult to deploy. We introduce Depth-Structured Music Recurrence (DSMR), a recurrent long-context Transformer for full-piece symbolic music modeling that extends context beyond fixed-length excerpts via segment-level recurrence with detached cross-segment states, featuring a layer-wise memory-horizon schedule that budgets recurrent KV states across depth. DSMR is trained in a single left-to-right pass over each complete composition, akin to how a musician experiences it from beginning to end, while carrying recurrent cross-segment states forward. Within this recurrent framework, we systematically study how depth-wise horizon allocations affect optimization, best-checkpoint perplexity, and efficiency. By allocating different history-window lengths across layers while keeping the total recurrent-state budget fixed, DSMR creates depth-dependent temporal receptive fields within a recurrent attention stack without reducing compute depth. Our main instantiation is a two-scale DSMR schedule that allocates long history windows to lower layers and a uniform short window to the remaining layers. Experiments on the piano performance dataset MAESTRO demonstrate that two-scale DSMR provides a practical quality--efficiency recipe for full-length long-context symbolic music modeling with recurrent attention under limited computational resources.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a novel approach to long-context modeling in symbolic music generation through the Depth-Structured Music Recurrence framework, effectively balancing computational efficiency with the need for extensive contextual information. The comprehensive methodology and rigorous experimental evaluation contribute to its significance in advancing the field of machine learning for music generation.
Methodology
The proposed Depth-Structured Music Recurrence (DSMR) framework innovatively addresses the challenges of long-context modeling in symbolic music generation by implementing a recurrent long-context Transformer that utilizes segment-level recurrence with detached cross-segment states. The methodology is well-structured, allowing for depth-dependent temporal receptive fields by varying memory horizons across layers. This approach is particularly relevant for resource-constrained environments, as it balances computational efficiency with the need for extensive contextual information in music generation.
Results & evaluation
The experiments conducted on the MAESTRO dataset are rigorous and well-documented, demonstrating the effectiveness of the two-scale DSMR model in achieving lower perplexity compared to other methods under similar memory constraints. The evaluation metrics, including perplexity and efficiency (tokens processed per second, peak memory usage), provide a comprehensive view of the model's performance. The comparative analysis with other models, including full-attention references, adds credibility to the findings.
Reproducibility
The paper provides sufficient details regarding the experimental setup, model architecture, and training protocols, which enhances reproducibility. However, the absence of a publicly available code repository or demo limits the ease with which other researchers can replicate the results.
Limitations
The study acknowledges certain limitations, such as the potential impact of the chosen model scale and memory settings on performance. Additionally, the findings may not generalize to all long-context domains, and the exploration of the design space for memory retention and gating could be expanded.
Broader impact
The implications of this research are significant for the field of symbolic music generation, particularly in enabling more efficient models that can operate on consumer-grade hardware. This advancement could facilitate real-time applications in music composition and performance, making sophisticated music generation tools more accessible to creators. The paper presents a novel approach to long-context modeling in symbolic music generation through the Depth-Structured Music Recurrence framework, effectively balancing computational efficiency with the need for extensive contextual information. The comprehensive methodology and rigorous experimental evaluation contribute to its significance in advancing the field of machine learning for music generation.
Author of this paper? Send a correction.