FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System
Abstract
We present FireRedASR2S, a state-of-the-art industrial-grade all-in-one automatic speech recognition (ASR) system. It integrates four modules in a unified pipeline: ASR, Voice Activity Detection (VAD), Spoken Language Identification (LID), and Punctuation Prediction (Punc). All modules achieve SOTA performance on the evaluated benchmarks: FireRedASR2: An ASR module with two variants, FireRedASR2-LLM (8B+ parameters) and FireRedASR2-AED (1B+ parameters), supporting speech and singing transcription for Mandarin, Chinese dialects and accents, English, and code-switching. Compared to FireRedASR, FireRedASR2 delivers improved recognition accuracy and broader dialect and accent coverage. FireRedASR2-LLM achieves 2.89% average CER on 4 public Mandarin benchmarks and 11.55% on 19 public Chinese dialects and accents benchmarks, outperforming competitive baselines including Doubao-ASR, Qwen3-ASR, and Fun-ASR. FireRedVAD: An ultra-lightweight module (0.6M parameters) based on the Deep Feedforward Sequential Memory Network (DFSMN), supporting streaming VAD, non-streaming VAD, and multi-label VAD (mVAD). On the FLEURS-VAD-102 benchmark, it achieves 97.57% frame-level F1 and 99.60% AUC-ROC, outperforming Silero-VAD, TEN-VAD, FunASR-VAD, and WebRTC-VAD. FireRedLID: An Encoder-Decoder LID module supporting 100+ languages and 20+ Chinese dialects and accents. On FLEURS (82 languages), it achieves 97.18% utterance-level accuracy, outperforming Whisper and SpeechBrain. FireRedPunc: A BERT-style punctuation prediction module for Chinese and English. On multi-domain benchmarks, it achieves 78.90% average F1, outperforming FunASR-Punc (62.77%). To advance research in speech processing, we release model weights and code at https://github.com/FireRedTeam/FireRedASR2S.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces FireRedASR2S, a state-of-the-art all-in-one ASR system that integrates multiple modules to enhance speech recognition accuracy and robustness across various languages and dialects. The technical contributions are substantial, particularly in the context of modular design and comprehensive evaluation, making it a valuable addition to the field of automatic speech recognition.
Methodology
The paper presents a comprehensive and modular architecture for an automatic speech recognition system that integrates multiple components (ASR, VAD, LID, Punc) into a unified pipeline. The methodology is well-structured, leveraging state-of-the-art techniques such as the Encoder-Decoder architecture for LID and BERT-style models for punctuation prediction. The use of a large and diverse training corpus enhances the model's generalization capabilities across various dialects and languages, which is a significant methodological strength.
Results & evaluation
The experimental evaluation is robust, with extensive benchmarking against multiple public datasets, demonstrating superior performance across all components. The results are clearly presented, showing improvements in character error rates and other relevant metrics compared to existing systems. The use of human-annotated data for training VAD is particularly noteworthy, as it enhances the reliability of segmentation in real-world applications.
Reproducibility
The authors have committed to open-sourcing their model weights and code, which is a positive step towards reproducibility. However, the paper could benefit from more detailed descriptions of the training processes and hyperparameter settings to facilitate replication of results by other researchers.
Limitations
While the system shows impressive performance, it may still face challenges in extremely noisy environments or with highly accented speech that diverges from the training data. Additionally, the reliance on large-scale training data may limit accessibility for smaller institutions or researchers with fewer resources.
Broader impact
The FireRedASR2S system has significant implications for various applications, including real-time transcription services, multilingual communication tools, and accessibility technologies. Its modular design allows for flexible deployment in diverse settings, potentially advancing the state of the art in speech recognition technology. The paper introduces FireRedASR2S, a state-of-the-art all-in-one ASR system that integrates multiple modules to enhance speech recognition accuracy and robustness across various languages and dialects. The technical contributions are substantial, particularly in the context of modular design and comprehensive evaluation, making it a valuable addition to the field of automatic speech recognition.
Author of this paper? Send a correction.