MDM-ASR: Bridging Accuracy and Efficiency in ASR with Diffusion-Based Non-Autoregressive Decoding
Abstract
In sequence-to-sequence Transformer ASR, autoregressive (AR) models achieve strong accuracy but suffer from slow decoding, while non-autoregressive (NAR) models enable parallel decoding at the cost of degraded performance. We propose a principled NAR ASR framework based on Masked Diffusion Models to reduce this gap. A pre-trained speech encoder is coupled with a Transformer diffusion decoder conditioned on acoustic features and partially masked transcripts for parallel token prediction. To mitigate the training-inference mismatch, we introduce Iterative Self-Correction Training that exposes the model to its own intermediate predictions. We also design a Position-Biased Entropy-Bounded Confidence-based sampler with positional bias to further boost results. Experiments across multiple benchmarks demonstrate consistent gains over prior NAR models and competitive performance with strong AR baselines, while retaining parallel decoding efficiency.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The MDM-ASR framework presents a significant advancement in bridging the gap between accuracy and efficiency in ASR systems. By innovatively applying masked diffusion models and iterative self-correction training, the authors provide a compelling solution that enhances both the performance and practicality of non-autoregressive speech recognition.
Methodology
The proposed MDM-ASR framework introduces a novel approach to non-autoregressive (NAR) automatic speech recognition (ASR) by leveraging masked diffusion models. The methodology effectively combines a pre-trained speech encoder with a diffusion-based decoder that utilizes iterative self-correction training (ISCT) to align training and inference processes. This approach addresses the limitations of traditional CTC models and autoregressive methods, particularly in terms of decoding efficiency and accuracy. The introduction of position-biased entropy-bounded confidence-based samplers further enhances decoding performance, showcasing a thoughtful integration of existing techniques into a cohesive framework.
Results & evaluation
The experimental validation is robust, involving multiple benchmark datasets including LibriSpeech, Earnings22, AMI, and VoxPopuli, demonstrating the model's competitive performance against strong autoregressive baselines. The results indicate that MDM-ASR not only achieves lower word error rates (WER) but also maintains superior decoding efficiency, particularly in longer sequences. Comprehensive ablation studies provide insights into the effectiveness of the proposed methods, reinforcing the validity of the experimental design.
Reproducibility
While the paper mentions plans to open source the code and hyperparameters, the current lack of a publicly available implementation limits immediate reproducibility. The detailed description of the model architecture and training procedures, however, suggests that other researchers could replicate the study with sufficient effort.
Limitations
The paper acknowledges limitations regarding the evaluation scope, which is confined to a subset of datasets, potentially limiting the generalizability of the findings. Additionally, the reliance on specific design choices may restrict the model's adaptability to varied ASR applications. Future work is needed to explore alternative architectures and broader dataset evaluations.
Broader impact
The advancements presented in this research have significant implications for real-time ASR applications, particularly in environments where efficiency is critical, such as virtual assistants and transcription services. The ability to achieve competitive accuracy with faster decoding times could enhance user experience and expand the applicability of ASR technologies across diverse domains. The MDM-ASR framework presents a significant advancement in bridging the gap between accuracy and efficiency in ASR systems. By innovatively applying masked diffusion models and iterative self-correction training, the authors provide a compelling solution that enhances both the performance and practicality of non-autoregressive speech recognition.
Author of this paper? Send a correction.