Decoding Strategies for Diffusion-Based ASR: A Systematic Evaluation of Confidence-Based Thresholding
Abstract
While LLM-based Automatic Speech Recognition (ASR) achieves high accuracy, its speed is limited by sequential autoregressive decoding. Diffusion Language Models (DLMs) offer a parallel alternative, yet their decoding strategies remain under-explored in ASR contexts. This paper analyzes three decoding schemes for DLM-based ASR: fixed-number, static confidence threshold, and dynamic confidence threshold. We propose measuring round-wise accuracy using Negative Log-Likelihood-based uncertainty as a proxy for decoding progress. Our results show that both threshold-based strategies significantly outperform fixed-number schemes in accuracy and speed. We attribute this to a property unique to ASR: most tokens reach high confidence early, allowing reliable ones to be harvested aggressively while leaving only difficult tokens for later rounds. Notably, the static-threshold strategy matches the accuracy of autoregressive decoding while offering superior efficiency.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the systematic evaluation of decoding strategies for DLM-based ASR, revealing that static and dynamic thresholding significantly enhance accuracy and speed compared to fixed-number decoding. This work provides a crucial step towards optimizing ASR systems, particularly in leveraging the unique properties of DLMs for improved performance.
Methodology
The paper presents a systematic evaluation of decoding strategies for DLM-based ASR, comparing fixed-number, static threshold, and dynamic threshold approaches. The methodology is well-structured, utilizing Negative Log-Likelihood (NLL) as a measure of uncertainty, which is a novel approach in this context. The authors effectively analyze the performance of each strategy in terms of accuracy and speed, providing a clear rationale for their findings. However, the reliance on a single baseline model (Whisper-LLaDA) may limit the generalizability of the results.
Results & evaluation
The experiments are comprehensive, utilizing the LibriSpeech dataset and focusing on various hyperparameters for each decoding strategy. The evaluation metrics, including Word Error Rate (WER) and Real-Time Factor (RTF), are appropriate for assessing the performance of ASR systems. The results indicate that threshold-based strategies significantly outperform fixed-number schemes, which is a valuable contribution to the field. However, the paper could benefit from additional experiments on diverse datasets to validate the findings further.
Reproducibility
The paper provides sufficient details on the experimental setup, including the training process and evaluation metrics. However, the absence of code or a project URL limits reproducibility. Future work should include sharing the implementation to facilitate validation by other researchers.
Limitations
The study is limited to clean read English speech from the LibriSpeech test-clean set, which may not fully represent the challenges of noisy or spontaneous speech. Additionally, the findings may not generalize to multilingual ASR systems, as the confidence distribution could vary significantly across different languages and contexts.
Broader impact
The findings have significant implications for the development of more efficient ASR systems, particularly in applications requiring real-time processing. By demonstrating the effectiveness of threshold-based decoding strategies, this work could influence future research directions in ASR and related fields, potentially leading to advancements in speech technology and accessibility. The main contribution of this paper is the systematic evaluation of decoding strategies for DLM-based ASR, revealing that static and dynamic thresholding significantly enhance accuracy and speed compared to fixed-number decoding. This work provides a crucial step towards optimizing ASR systems, particularly in leveraging the unique properties of DLMs for improved performance.
Author of this paper? Send a correction.