Explainable Multi-Modal Deep Learning for Automatic Detection of Lung Diseases from Respiratory Audio Signals
Abstract
Respiratory diseases remain major global health challenges, and traditional auscultation is often limited by subjectivity, environmental noise, and inter-clinician variability. This study presents an explainable multimodal deep learning framework for automatic lung-disease detection using respiratory audio signals. The proposed system integrates two complementary representations: a spectral-temporal encoder based on a CNN-BiLSTM Attention architecture, and a handcrafted acoustic-feature encoder capturing physiologically meaningful descriptors such as MFCCs, spectral centroid, spectral bandwidth, and zero-crossing rate. These branches are combined through late-stage fusion to leverage both data-driven learning and domain-informed acoustic cues. The model is trained and evaluated on the Asthma Detection Dataset Version 2 using rigorous preprocessing, including resampling, normalization, noise filtering, data augmentation, and patient-level stratified partitioning. The study achieved strong generalization with 91.21% accuracy, 0.899 macro F1-score, and 0.9866 macro ROC-AUC, outperforming all ablated variants. An ablation study confirms the importance of temporal modeling, attention mechanisms, and multimodal fusion. The framework incorporates Grad-CAM, Integrated Gradients, and SHAP, generating interpretable spectral, temporal, and feature-level explanations aligned with known acoustic biomarkers to build clinical transparency. The findings demonstrate the framework's potential for telemedicine, point-of-care diagnostics, and real-world respiratory screening.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This study presents a novel explainable multimodal deep learning framework for automatic lung disease detection from respiratory audio signals, addressing critical challenges in traditional auscultation methods. The integration of deep learning with handcrafted features and explainable AI techniques represents a significant advancement in the field, with the potential to improve clinical outcomes through enhanced diagnostic accuracy and transparency.
Methodology
The proposed methodology integrates a hybrid deep learning architecture combining CNN, BiLSTM, and attention mechanisms with handcrafted acoustic features, which is innovative in the context of respiratory sound analysis. The late-stage fusion approach effectively leverages both data-driven and domain-informed representations, enhancing the model's robustness and interpretability. The incorporation of explainable AI techniques such as Grad-CAM, Integrated Gradients, and SHAP adds significant value by providing clinical transparency, which is often lacking in deep learning applications in healthcare.
Results & evaluation
The experiments are well-structured, utilizing a publicly available dataset with a comprehensive evaluation strategy that includes accuracy, F1-score, and ROC-AUC metrics. The ablation study is particularly noteworthy, as it rigorously tests the contributions of various components of the model, confirming the importance of multimodal fusion and attention mechanisms. The reported results demonstrate strong generalization capabilities across different respiratory conditions, indicating the model's practical applicability.
Reproducibility
The paper outlines a clear training strategy, including hyperparameter settings, data preprocessing steps, and evaluation metrics, which enhances reproducibility. However, the absence of a publicly accessible code repository limits the ability for others to replicate the study fully.
Limitations
While the study presents a robust framework, it relies on a single dataset, which may limit the generalizability of the findings. The model's performance on underrepresented classes, such as Bronchial sounds, suggests that further refinement may be necessary to improve classification accuracy across all categories. Additionally, the lack of a demo or project URL restricts practical engagement with the research.
Broader impact
The framework has significant implications for telemedicine and point-of-care diagnostics, potentially improving early detection and management of respiratory diseases. By enhancing the interpretability of AI models in clinical settings, this work contributes to building trust in automated diagnostic systems, which is crucial for their acceptance in healthcare. This study presents a novel explainable multimodal deep learning framework for automatic lung disease detection from respiratory audio signals, addressing critical challenges in traditional auscultation methods. The integration of deep learning with handcrafted features and explainable AI techniques represents a significant advancement in the field, with the potential to improve clinical outcomes through enhanced diagnostic accuracy and transparency.
Author of this paper? Send a correction.