Spark-Audio 1.0 Technical Report
Abstract
This paper introduces Spark-Audio 1.0, a speech foundation model trained on domestically produced computing infrastructure. The model accepts text and speech inputs and generates text outputs. It supports multilingual (99 languages) and multi-dialect (202 dialects) speech recognition, speech translation, audio perception and understanding, and both text- and speech-based dialogue, marking a shift from merely ``hearing clearly'' to genuinely ``understanding'' speech. Architecturally, Spark-Audio 1.0 couples a 0.65B-parameter dense audio encoder with a 30B-A3B Mixture-of-Experts (MoE) large language model (LLM). The audio encoder is pretrained with representation-enhanced BEST-RQ and then refined through multi-task supervised training. The audio--LLM is trained through staged pretraining, followed by post-training that combines supervised fine-tuning with on-policy distillation. In total, the model is trained on 13 million hours of audio data together with large-scale text data. We evaluate Spark-Audio 1.0 on mainstream audio and text benchmarks, as well as on in-house test sets drawn from real-world application scenarios. Despite using substantially less audio training data than Qwen3.5-Omni-Flash, Spark-Audio 1.0 achieves comparable performance on most audio and text tasks, outperforms it in multilingual speech recognition, spoken dialogue, and real-world application scenarios, and approaches the performance of larger closed-source models. Nevertheless, there remains room for improvement in instruction following and complex audio reasoning. Finally, the successful training of Spark-Audio 1.0 on domestic hardware, including Huawei Ascend 910B and Hygon BW1000 chips, demonstrates the feasibility of a domestic technical pathway for building speech foundation models.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution is the demonstration of a competitive, data-efficient speech foundation model trained entirely on domestic Chinese hardware. The paper provides a detailed technical report on the Spark-Audio 1.0 architecture, highlighting a novel representation-enhanced pretraining strategy and a unified streaming/non-streaming encoder design that achieves strong multilingual performance with significantly less data than comparable competitors.
Methodology
The paper proposes a unified speech foundation model architecture combining a 0.65B dense audio encoder (based on Whisper-Large v3) with a 30B-A3B MoE LLM. The core methodological contribution is the "representation-enhanced BEST-RQ" pretraining strategy, which uses frozen semantic representations from a multilingual ASR teacher to generate discrete targets for the student encoder, aiming to inject semantic priors without additional trainable tokenizers. The training pipeline is staged: self-supervised pretraining, supervised ASR fine-tuning, and multi-task perception training (emotion, gender, etc.). The audio-LLM integration uses a staged recipe including adapter alignment, joint optimization, and long-context extension, followed by SFT and on-policy distillation (OPD) to preserve text capabilities. The unified streaming/non-streaming encoder design via dynamic chunk training is a practical engineering contribution.
Results & evaluation
The model is evaluated on 99 languages and 202 dialects, covering ASR, translation, audio understanding, and dialogue. The paper claims comparable performance to Qwen3.5-Omni-Flash with significantly less data (13M hours vs. implied larger scale for Qwen) and outperforms it in multilingual recognition and real-world scenarios. It also approaches larger closed-source models. However, the evaluation relies heavily on "in-house test sets" which are not publicly available, limiting independent verification. The comparison with Qwen3.5-Omni-Flash is the primary baseline, but details on the specific data scale of the competitor are somewhat vague ("substantially less").
Reproducibility
Reproducibility is low for the general community. The model is trained on specific domestic hardware (Huawei Ascend 910B, Hygon BW1000), and the code/weights are not released in the paper. The "in-house" benchmarks are proprietary. While the architectural details (Whisper-based encoder, MoE LLM) are standard, the specific data curation (13M hours) and the "representation-enhanced" teacher setup are not fully detailed enough for exact replication without access to iFlytek's internal resources.
Limitations
The paper explicitly acknowledges room for improvement in instruction following and complex audio reasoning. The reliance on proprietary in-house benchmarks makes it difficult for the community to verify the claimed superiority over Qwen3.5-Omni-Flash. The "domestic hardware" aspect, while strategically important, limits the immediate technical applicability for researchers using standard CUDA stacks. The text-only output modality restricts it to understanding rather than generation (speech-to-speech).
Broader impact
The paper demonstrates the feasibility of training large-scale speech foundation models on non-NVIDIA hardware, which is significant for the Chinese AI ecosystem and potentially other regions with hardware restrictions. It contributes to the trend of data-efficient training in audio-LLMs. The support for 202 dialects is a substantial resource for multilingual NLP research, assuming the model or data becomes available. The main contribution is the demonstration of a competitive, data-efficient speech foundation model trained entirely on domestic Chinese hardware. The paper provides a detailed technical report on the Spark-Audio 1.0 architecture, highlighting a novel representation-enhanced pretraining strategy and a unified streaming/non-streaming encoder design that achieves strong multilingual performance with significantly less data than comparable competitors.
Author of this paper? Send a correction.