Scalable Keyword Spotting via Modular Network Expansion
Abstract
Keyword spotting (KWS) models on embedded devices often need to add new keywords after deployment, but updates are difficult when original training data are unavailable and regressions on existing triggers are unacceptable. At a fixed operating point, our method reduces average new-keyword false reject rate (FRR) from 6.46 to 4.37 versus a parameter-matched separate-model baseline and outperforms parameter-efficient tuning baselines (adapters, LoRA), while using fewer multiply-accumulate operations (MACs) under the same added-parameter budget ($\leq$10k): 16.34M vs 18.45M/20.52M. We achieve this via parameter-capped modular expansion: the base network, including batch-normalization statistics and the core classifier, is frozen, and only a lightweight expansion branch with a separate new-keyword head is trained, preserving core logits, shipped outputs, and thresholds for existing keywords.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a practical and effective method for modular expansion of embedded KWS models, achieving strict non-regulation and improved new-keyword detection through a frozen-base, parallel-branch architecture.
Methodology
The paper proposes a "modular network expansion" technique for Keyword Spotting (KWS) on embedded devices. The core innovation is architectural: it freezes the entire base SVDF network (including Batch Normalization statistics) and attaches a lightweight, trainable expansion branch with a separate classifier head for new keywords. This design ensures a strict non-regression guarantee for existing keywords because the core logits remain identical to the pre-deployment model. The method is compared against parameter-efficient fine-tuning (LoRA, Adapters), ensemble methods, and continual learning (EWC). The approach is pragmatic and well-suited for the specific constraints of embedded deployment (no original data, strict latency/parameter budgets). While the concept of "parallel heads" or "expert routing" is not new in the broader ML landscape (e.g., Mixture of Experts, Progressive Neural Networks), its specific application to strict non-regression KWS updates with frozen BN layers is a solid, incremental engineering contribution. It lacks deep theoretical novelty but offers a robust practical solution.
Results & evaluation
The experiments are conducted on Google Speech Commands v2 (GSC) using five held-out keyword pairs. The evaluation metric is macro False Reject Rate (FRR) at a fixed False Accept Rate (FAR). The results show that the proposed method significantly outperforms the Ensemble baseline (6.46% vs 4.37% FRR) and other parameter-efficient methods (LoRA, Adapters) under a 10k parameter budget. The ablation study on expansion depth is useful, identifying that tapping 4 blocks is optimal. The comparison with Full Fine-tuning highlights the catastrophic forgetting problem, and EWC is shown to be less effective without replay data. The use of Mozilla Common Voice for negative samples is a good practice for realistic FAR calibration. The results are statistically significant (95% confidence intervals provided). However, the evaluation is limited to GSC, which is a relatively clean, controlled dataset. Real-world embedded KWS faces more challenges with noise, channel variation, and long-tail distributions.
Reproducibility
The paper provides sufficient detail regarding the architecture (SVDF, BN, Hard-Swish), training setup (Adam, cosine LR, SpecAugment), and evaluation protocol (FAR calibration, 8 runs). The parameter budget (10k) and MAC counts are specified. The code is not explicitly linked, but the description is detailed enough for reproduction. The use of standard datasets (GSC, Common Voice) aids reproducibility.
Limitations
The primary limitation is the reliance on GSC, which may not reflect the complexity of real-world embedded audio environments. The method assumes that the base model's features are sufficiently generalizable to the new keywords via the expansion branch; if the new keywords are phonetically very distinct from the core set, performance might degrade. The "core-first" decision rule introduces a slight latency overhead (checking core first, then new), although the paper argues this is negligible or manageable. The method is specific to KWS and may not generalize easily to other incremental learning tasks without modification.
Broader impact
This work has significant practical impact for the deployment of always-on voice assistants on resource-constrained devices. By enabling safe, data-efficient updates to keyword inventories, it addresses a major pain point in product lifecycle management for IoT devices. It reduces the need for retraining on large datasets and mitigates the risk of degrading user experience for existing features. The paper presents a practical and effective method for modular expansion of embedded KWS models, achieving strict non-regulation and improved new-keyword detection through a frozen-base, parallel-branch architecture.
Author of this paper? Send a correction.