StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks
Abstract
Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
StethoLM establishes a foundation for instruction-following AI systems in clinical auscultation, demonstrating that audio-language models can move beyond narrow classification to support the diverse reasoning tasks clinicians perform. The combination of domain-specialized training with interpretable natural language interfaces represents a meaningful advancement in medical AI applications, particularly in the context of cardiopulmonary health.
Methodology
The methodology presented in this paper is robust and innovative, combining audio encoding with a medical language model to create an audio-language model specifically tailored for cardiopulmonary auscultation tasks. The introduction of StethoBench as a benchmark for training and evaluating models on diverse clinical tasks is a significant contribution, as it moves beyond traditional classification paradigms. The multi-stage training approach, which includes supervised fine-tuning and direct preference optimization, demonstrates a thoughtful design aimed at improving model performance and robustness.
Results & evaluation
The experiments conducted are comprehensive, utilizing a large dataset of 77,027 instruction-response pairs derived from 16,125 clinical recordings. The evaluation metrics, including BERTScore and clinical accuracy judged by a large language model, provide a nuanced understanding of the model's performance across various clinical tasks. The results indicate substantial improvements over general-purpose models, showcasing the effectiveness of the specialized training approach.
Reproducibility
The paper provides detailed implementation details, including model architecture, training methodology, and evaluation metrics, which enhance reproducibility. The use of publicly available datasets and the provision of a GitHub repository for the model further support the reproducibility of the research.
Limitations
The paper acknowledges several limitations, including the reliance on single reference annotations, potential biases in the aggregated datasets, and the lack of expert clinician validation for the clinical accuracy assessments. Additionally, the model's occasional failure to appropriately refuse responses without audio input highlights areas for improvement.
Broader impact
The development of StethoLM has significant implications for clinical practice, particularly in enhancing the capabilities of auscultation in settings with limited access to experienced clinicians. By providing interpretable, instruction-driven outputs, this model could facilitate better decision-making and improve patient outcomes in cardiopulmonary care. StethoLM establishes a foundation for instruction-following AI systems in clinical auscultation, demonstrating that audio-language models can move beyond narrow classification to support the diverse reasoning tasks clinicians perform. The combination of domain-specialized training with interpretable natural language interfaces represents a meaningful advancement in medical AI applications, particularly in the context of cardiopulmonary health.
Author of this paper? Send a correction.