AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Robust Phonocardiogram Classification
Abstract
Biomedical audio signals, such as phonocardiograms (PCG), are inherently rhythmic and contain diagnostic information in both their spectral (tonal) and temporal domains. Standard 2D spectrograms provide rich spectral features but compromise the phase information and temporal precision of the 1D waveform. We propose AudioFuse, an architecture that simultaneously learns from both complementary representations to classify PCGs. To mitigate the overfitting risk common in fusion models, we integrate a custom, wide-and-shallow Vision Transformer (ViT) for spectrograms with a shallow 1D CNN for raw waveforms. On the PhysioNet 2016 dataset, AudioFuse achieves a state-of-the-art competitive ROC-AUC of 0.8608 when trained from scratch, outperforming its spectrogram (0.8066) and waveform (0.8223) baselines. Moreover, it demonstrates superior robustness to domain shift on the challenging PASCAL dataset, maintaining an ROC-AUC of 0.7181 while the spectrogram baseline collapses (0.4873). Fusing complementary representations thus provides a strong inductive bias, enabling the creation of efficient, generalizable classifiers without requiring large-scale pre-training.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of AudioFuse, a hybrid ViT-1D CNN architecture that successfully fuses spectral and temporal information for robust phonocardiogram classification. This work represents a significant advancement in the methodology for analyzing biomedical audio signals, combining innovative architectural design with rigorous experimental validation, thereby enhancing the potential for real-world applications in healthcare diagnostics.
Methodology
The proposed AudioFuse architecture effectively combines a Vision Transformer (ViT) for spectrograms with a 1D CNN for waveforms, addressing the limitations of single-modality approaches in phonocardiogram classification. The late fusion strategy allows for independent feature extraction, preserving the strengths of both modalities while minimizing the risk of overfitting. The design choices are well-justified, and the architecture is lightweight, making it suitable for practical applications in biomedical signal processing.
Results & evaluation
The experiments are robust, utilizing two distinct datasets (PhysioNet 2016 and PASCAL) to evaluate both in-domain performance and generalization capabilities. The results demonstrate a significant improvement over baseline models in terms of ROC-AUC, accuracy, and F1-score, indicating the effectiveness of the dual-branch architecture. The inclusion of ablation studies provides additional insights into the performance of different fusion strategies.
Reproducibility
The paper provides sufficient implementation details, including the architecture specifications, training protocols, and hyperparameters, which enhances reproducibility. The availability of the source code and pretrained model weights on GitHub further supports this aspect, allowing other researchers to replicate the findings.
Limitations
While the model shows promising results, the reliance on specific datasets may limit the generalizability of the findings to other types of biomedical audio signals. Additionally, the performance on the PASCAL dataset indicates that there may still be challenges in domain adaptation that need to be addressed.
Broader impact
The AudioFuse architecture has the potential to significantly advance the field of biomedical audio classification, enabling more accurate and robust diagnostic tools for non-invasive health monitoring. Its applicability extends beyond phonocardiograms to other audio signals in healthcare, such as respiratory sounds, which could lead to improved patient outcomes. The main contribution of this paper is the introduction of AudioFuse, a hybrid ViT-1D CNN architecture that successfully fuses spectral and temporal information for robust phonocardiogram classification. This work represents a significant advancement in the methodology for analyzing biomedical audio signals, combining innovative architectural design with rigorous experimental validation, thereby enhancing the potential for real-world applications in healthcare diagnostics.
Author of this paper? Send a correction.