Apriel Voice 27B Preview: An Open Model for Speech, Reasoning, and Tool Use
Abstract
A capable voice assistant must do more than recognize words; it must understand requests, reason over context, sustain conversations, and act through tools. We present Apriel Voice 27B Preview, an open speech-language model that unifies these capabilities within a shared reasoning backbone while retaining text and image input modalities. Built on the post-trained Gemma 4 26B-A4B checkpoint, Apriel Voice directly processes speech and generates both responses and tool calls without requiring a separate dialogue model. We introduce a two-stage training procedure to adapt the model to speech. The first stage establishes the speech interface, while the second jointly adapts the projector and backbone through supervised fine-tuning, text replay, and source-dependent cross-entropy and forward Kullback--Leibler divergence supervision. Automatically constructed speech-conditioned dialogues provide conversational training data, while local teacher serving, overlapping teacher and student computation, and fused full-vocabulary losses enable efficient online distillation. In thinking mode, Apriel Voice achieves a score of 98.90 on Big Bench Audio, 81.13 on Speech-BFCL, and a task success rate of 65.51% on $τ$-Voice, with a mean latency of 1.91 seconds. It also achieves 51.27% English EVA accuracy and 74.13% experience pass@1. Finally, we deploy Apriel Voice 27B Preview on a custom serving platform to support interactive voice interactions.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
Apriel Voice 27B Preview is an open speech-language model that unifies speech understanding, reasoning, and tool use within a shared backbone. The paper presents a rigorous two-stage adaptation recipe combining projector training with joint fine-tuning using source-dependent CE/FKL supervision, demonstrating strong performance in agentic speech tasks while retaining text and image capabilities.
Methodology
The paper proposes a two-stage training procedure to adapt a text-and-image LLM (Gemma 4 26B-A4B) into a speech-language model. Stage 1 freezes the backbone and trains a speech projector (Whisper encoder + MLP) on transcription, translation, and continuation tasks. Stage 2 jointly fine-tunes the projector and backbone using a mixture of speech tasks, synthetic speech-conditioned dialogues, tool interactions, and text replay. A key technical contribution is the source-dependent loss function combining Cross-Entropy (CE) for task targets and Forward Kullback-Leibler (FKL) divergence from a frozen teacher model to preserve original capabilities. The system also details an efficient training infrastructure using local teacher serving, overlapping computation, and fused full-vocabulary loss kernels. While the individual components (projector, distillation, SFT) are established, the specific combination for agentic speech tasks and the engineering optimizations for online distillation are well-executed.
Results & evaluation
The evaluation is comprehensive, covering foundational speech metrics (WER, BLEU), reasoning (Big Bench Audio, GSM8K), instruction following (IFEval), and agentic task completion (EVA, τ-Voice, EnterpriseOps-Gym). The model demonstrates strong performance, particularly in reasoning (98.90 on BBA) and tool use (81.13 on BFCL), outperforming other open-source 2-stage models. However, the reliance on synthetic data for conversation training and the lack of ablation studies isolating the impact of specific training components (e.g., FKL vs. CE, specific data mixes) limit the depth of the empirical analysis. The comparison with proprietary systems is competitive but relies on different evaluation harnesses and simulators, making direct comparison difficult.
Reproducibility
The paper provides detailed architectural specifications (projector dimensions, parameter counts) and training hyperparameters (learning rates, batch sizes, GPU configuration). The release of an interactive demo and the use of open-source base models (Gemma, Whisper) enhance reproducibility. However, the synthetic data generation pipeline and specific internal datasets are not fully released, and the "thinking mode" implementation details are not fully specified, which may hinder exact reproduction of the reported reasoning scores.
Limitations
The model is primarily optimized for English, with limited multilingual speech understanding. The evaluation lacks paired significance tests against the base model, making it hard to quantify the exact cost of adaptation on text/image capabilities. The synthetic data pipeline may propagate errors from the TTS or LLM generators. The paper acknowledges that frozen visual components do not guarantee unchanged image reasoning, yet the drop in some vision benchmarks suggests potential degradation.
Broader impact
This work contributes to the development of open-source speech-language models capable of agentic tasks, which is a critical area for enterprise and consumer applications. By demonstrating that a single backbone can handle speech, text, images, and tool calls, it supports the trend toward unified multimodal agents. The efficient training techniques for online distillation are valuable for the broader community adapting large models to new modalities. Apriel Voice 27B Preview is an open speech-language model that unifies speech understanding, reasoning, and tool use within a shared backbone. The paper presents a rigorous two-stage adaptation recipe combining projector training with joint fine-tuning using source-dependent CE/FKL supervision, demonstrating strong performance in agentic speech tasks while retaining text and image capabilities.
Author of this paper? Send a correction.