SPECTRA: Subspace-Preserving Embedding Calibration, Transport, and Replay for Fully Few-Shot Class-Incremental Audio Classification
Abstract
Fully few-shot class-incremental audio classification (FFCAC) requires recognizing new sound classes from only a handful of labeled examples per session, without forgetting previously learned classes and without any large base dataset. Existing methods typically freeze a pre-trained audio--language encoder and classify with point prototypes, but they suffer from significant performance degradation throughout the sessions due to generic feature representations. We propose SPECTRA, a framework built on a frozen encoder which adds three components. (i) a lightweight trainable adapter that calibrates the generic embeddings to the task; (ii) subspace feature replay, an exemplar-free anti-forgetting scheme that replays old classes by sampling from the low-rank subspace of their stored features; and (iii) a transductive optimal-transport refinement of prototypes at test time. Our central finding is that the subspace structure of the replay diminishes forgetting and outperforms naive Gaussian replay of equal variance. On three FFCAC benchmarks (NSynth-100, FSC-89, LS-100), SPECTRA improves average accuracy and reduces forgetting over current state-of-the-art methods, and our ablations statistically validate each component.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
SPECTRA introduces a subspace-based exemplar-free replay mechanism and transductive optimal transport refinement to improve fully few-shot class-incremental audio classification, demonstrating consistent gains over the TAPE baseline on standard benchmarks. The technical contribution is solid and incremental, offering a practical solution to catastrophic forgetting in resource-constrained audio settings, though the novelty is moderate as it combines known techniques in a new configuration.
Methodology
The paper proposes SPECTRA, a framework for Fully Few-Shot Class-Incremental Audio Classification (FFCAC). It builds upon the TAPE baseline by introducing three key components: (1) a lightweight trainable residual-FFN adapter to calibrate frozen audio-language model (ALM) embeddings, (2) Subspace Feature Replay, an exemplar-free anti-forgetting mechanism that synthesizes old-class features by sampling from the low-rank subspace of stored support features, and (3) Transductive Optimal Transport (OT) for prototype refinement at test time. The methodology is technically sound, leveraging established concepts from continual learning (replay, adapters) and few-shot learning (prototypical networks, optimal transport) but applying them in a specific, constrained audio setting. The core novelty lies in the subspace replay mechanism, which claims to preserve feature geometry better than isotropic Gaussian replay. The integration of these components into a frozen-encoder pipeline is a coherent engineering contribution.
Results & evaluation
The authors evaluate SPECTRA on three benchmarks: NSynth-100, FSC-89, and LS-100. They report improvements in Average Accuracy (AA) and Performance Drop (PD) over the TAPE baseline. The results are presented in tables showing per-session accuracy. The ablation studies are comprehensive, isolating the effects of the adapter, the subspace structure vs. Gaussian replay, and the OT refinement. The statistical validation (50 seeds) adds credibility. However, the improvements, while consistent, are moderate (e.g., +4.8% AA on NSynth-100). The comparison is primarily against TAPE, which is the current SOTA for this specific setting, but broader comparisons with other FFCAC methods or general continual learning techniques adapted for audio are limited. The "harder protocol" experiment adds some robustness to the claims.
Reproducibility
The paper provides detailed descriptions of the components, including equations for the adapter, subspace sampling, and OT refinement. The experimental setup (datasets, protocols, hyperparameters like rank k=3, expansion r=3) is clearly stated. The use of a frozen PENGI encoder and TAPE's transform ensures a strong baseline for comparison. The code is not explicitly linked in the provided text, but the methodological details are sufficient for reproduction by experts in the field. The reliance on specific ALMs (PENGI) and the closed-form transform of TAPE limits direct comparability with methods using different backbones without re-implementation.
Limitations
The paper acknowledges that OT increases forgetting slightly (PD), which is compensated by subspace replay. The method is dependent on the quality of the frozen ALM embeddings; if the ALM is poor, the adapter may struggle to calibrate effectively. The subspace rank k is a hyperparameter that requires tuning, although the paper shows it is not highly sensitive in the tested range. The method assumes that the low-rank subspace is a sufficient statistic for the class distribution, which may not hold for highly complex or multimodal audio classes with very few shots. The evaluation is limited to three datasets, and generalization to other audio domains (e.g., speech, environmental sounds beyond the specific benchmarks) is not extensively explored.
Broader impact
This work contributes to the field of efficient and continual audio AI, which is crucial for real-world applications like assisted living, wildlife monitoring, and interactive systems where data is scarce and models must adapt over time without forgetting. By enabling exemplar-free learning, it addresses privacy and storage concerns. The techniques could be extended to other modalities or more complex continual learning scenarios. SPECTRA introduces a subspace-based exemplar-free replay mechanism and transductive optimal transport refinement to improve fully few-shot class-incremental audio classification, demonstrating consistent gains over the TAPE baseline on standard benchmarks. The technical contribution is solid and incremental, offering a practical solution to catastrophic forgetting in resource-constrained audio settings, though the novelty is moderate as it combines known techniques in a new configuration.
Author of this paper? Send a correction.