Samsone: A Family of Open Small Audio Language Models for On-Device Inference
Abstract
The success of Large Audio Language Models has driven the development of massive multimodal networks exceeding billions of parameters. However, the demand for privacy-preserving, low-latency processing has shifted focus toward Small Audio Language Models (SALMs) capable of on-device execution. In this paper, we introduce Samsone, a family of SALMs designed for edge computing. Our core model, Samsone-134M, establishes a new state-of-the-art for its size class across multiple benchmarks. We further explore the scaling laws of SALMs by introducing Samsone-99M and Samsone-356M. Despite their compact footprint, the Samsone family delivers performance competitive with models orders of magnitude larger. To foster open research and reproducibility, we train Samsone on publicly available data. We release the training code, model weights, mobile-optimized checkpoints and provide an open-source Android application to demonstrate real-time on-device inference of Samsone.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces Samsone, a family of small audio language models (SALMs) that achieve state-of-the-art performance for their size class by applying vocabulary reduction and depth pruning to standard encoder-decoder architectures. While the architectural components are not novel, the rigorous engineering of these models for on-device inference, combined with strong benchmark results that challenge the necessity of massive parameter counts for audio understanding, represents a significant practical contribution to the field of efficient multimodal AI.
Methodology
The methodology relies on a standard "encoder-projector-decoder" architecture, which is not novel in itself. The core technical contribution lies in the systematic application of size optimization techniques—specifically vocabulary reduction and depth pruning—to existing open-source components (Whisper-Tiny encoder and SmolLM2 backbone). The approach is pragmatic rather than architecturally groundbreaking. The use of a SEP token to handle multiple audio inputs is a sensible design choice for a general-purpose audio LM, but it is not a significant theoretical advance. The training procedure is straightforward single-stage fine-tuning, lacking the complex multi-stage pre-training or alignment strategies seen in larger LALMs.
Results & evaluation
The experimental evaluation is the strongest aspect of the paper. The authors provide a comprehensive comparison against both Large Audio Language Models (LALMs) and existing Small Audio Language Models (SALMs) on rigorous benchmarks like MMAU and MMAU-Pro. The results are compelling: Samsone-134M significantly outperforms previous SALMs (Mellow, Pengi) and approaches the performance of models 60x larger (Qwen2-Audio). The inclusion of on-device latency benchmarks on a specific hardware (Samsung Galaxy S25 Ultra) adds practical value, validating the "on-device" claim. However, the ablation study is somewhat limited, testing only a few alternative components without deep diving into the impact of specific training data mixes or the exact trade-offs of the pruning strategies beyond the final performance numbers.
Reproducibility
High. The authors explicitly state they release training code, model weights, and an Android application. The use of publicly available datasets (ReasonAQA, AudioSkillsXL) and open-source base models (Whisper, SmolLM2) ensures that the work can be easily reproduced by the community. The detailed description of the training setup (optimizer, learning rate, epochs) further supports reproducibility.
Limitations
The paper acknowledges that the models suffer from a loss of general-purpose linguistic capabilities due to extensive AQA fine-tuning. This is a common issue in domain-specific fine-tuning but limits the model's versatility. Additionally, the evaluation is heavily skewed towards multiple-choice and short-answer tasks (AQA); performance on open-ended generation or complex reasoning tasks beyond the benchmark scope is less clear. The lack of hardware-specific optimizations (NPU/GPU) means the reported latency may not represent the best possible performance on modern mobile devices.
Broader impact
This work is highly relevant to the growing field of edge AI and privacy-preserving audio processing. By demonstrating that sub-100M parameter models can achieve competitive performance with billion-parameter models, it lowers the barrier to entry for deploying advanced audio understanding on consumer devices. This has implications for real-time applications in healthcare, smart home assistants, and mobile interfaces where data privacy and latency are critical. The paper introduces Samsone, a family of small audio language models (SALMs) that achieve state-of-the-art performance for their size class by applying vocabulary reduction and depth pruning to standard encoder-decoder architectures. While the architectural components are not novel, the rigorous engineering of these models for on-device inference, combined with strong benchmark results that challenge the necessity of massive parameter counts for audio understanding, represents a significant practical contribution to the field of efficient multimodal AI.
Author of this paper? Send a correction.