LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation
Abstract
Vocal-to-accompaniment (V2A) generation, which aims to transform a raw vocal recording into a fully arranged accompaniment, inherently requires jointly addressing an accompaniment trilemma: preserving acoustic authenticity, maintaining global coherence with the vocal track, and producing dynamic orchestration across a full song. Existing open-source approaches typically make compromises among these goals. Continuous-latent generation models can capture long musical spans but often struggle to preserve fine-grained acoustic detail. In contrast, discrete autoregressive models retain local fidelity but suffer from unidirectional generation and error accumulation in extended contexts. We present LaDA-Band, an end-to-end framework that introduces Discrete Masked Diffusion to the V2A task. Our approach formulates V2A generation as Discrete Masked Diffusion, i.e., a global, non-autoregressive denoising formulation that combines the representational advantages of discrete audio codec tokens with full-sequence bidirectional context modeling. This design improves long-range structural consistency and temporal synchronization while preserving crisp acoustic details. Built on this formulation, LaDA-Band further introduces a dual-track prefix-conditioning architecture, an auxiliary replaced-token detection objective for weakly anchored accompaniment regions, and a two-stage progressive curriculum to scale Discrete Masked Diffusion to full-song vocal-to-accompaniment generation. Extensive experiments on both academic and real-world benchmarks show that LaDA-Band consistently improves acoustic authenticity, global coherence, and dynamic orchestration over existing baselines, while maintaining strong performance even without auxiliary reference audio. Codes and audio samples are available at https://github.com/Duoluoluos/TME-LaDA-Band .
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
LaDA-Band presents a novel approach to vocal-to-accompaniment generation through Discrete Masked Diffusion, significantly improving upon existing methods in terms of acoustic authenticity, coherence, and orchestration. The comprehensive methodology and rigorous experimental validation position this work as a meaningful contribution to the field of machine learning in audio generation.
Methodology
The methodology presented in LaDA-Band is innovative, leveraging Discrete Masked Diffusion to address the vocal-to-accompaniment generation problem. The dual-track prefix-conditioning architecture and the auxiliary replaced-token detection objective are significant contributions that enhance the model's ability to generate high-quality accompaniment while maintaining acoustic authenticity and global coherence. The two-stage progressive curriculum for training is a well-thought-out approach that allows the model to scale from short-form to full-song generation effectively.
Results & evaluation
The experiments conducted are extensive, comparing LaDA-Band against a variety of state-of-the-art baselines across multiple metrics. The results demonstrate consistent improvements in acoustic authenticity, global coherence, and dynamic orchestration, particularly under zero-shot conditions. The use of both objective metrics (like FAD and Onset F1) and subjective evaluations (like MOS) provides a comprehensive assessment of the model's performance.
Reproducibility
The paper provides detailed implementation specifics, including architecture choices, training procedures, and evaluation metrics, which enhance reproducibility. The availability of the code and audio samples further supports this aspect, allowing other researchers to replicate the study.
Limitations
While the paper acknowledges limitations such as dependency on the source separation and audio codec pipeline, it also notes challenges in fine-grained control over arrangement details and difficulties with certain stylistically free-form genres. These limitations suggest areas for future research and improvement.
Broader impact
The potential applications of LaDA-Band are significant, particularly in the music production industry, where automated accompaniment generation can streamline workflows for artists and producers. The framework's ability to generate high-quality music without extensive manual intervention could democratize music creation and enhance creative processes. LaDA-Band presents a novel approach to vocal-to-accompaniment generation through Discrete Masked Diffusion, significantly improving upon existing methods in terms of acoustic authenticity, coherence, and orchestration. The comprehensive methodology and rigorous experimental validation position this work as a meaningful contribution to the field of machine learning in audio generation.
Author of this paper? Send a correction.