Bias and Fairness in Self-Supervised Acoustic Representations for Cognitive Impairment Detection
Abstract
Speech-based detection of cognitive impairment (CI) offers a promising non-invasive approach for early diagnosis, yet performance disparities across demographic and clinical subgroups remain underexplored, raising concerns around fairness and generalizability. This study presents a systematic bias analysis of acoustic-based CI and depression classification using the DementiaBank Pitt Corpus. We compare traditional acoustic features (MFCCs, eGeMAPS) with contextualized speech embeddings from Wav2Vec 2.0 (W2V2), and evaluate classification performance across gender, age, and depression-status subgroups. For CI detection, higher-layer W2V2 embeddings outperform baseline features (UAR up to 80.6\%), but exhibit performance disparities; specifically, females and younger participants demonstrate lower discriminative power (\(AUC\): 0.769 and 0.746, respectively) and substantial specificity disparities (\(Δ_{spec}\) up to 18\% and 15\%, respectively), leading to a higher risk of misclassifications than their counterparts. These disparities reflect representational biases, defined as systematic differences in model performance across demographic or clinical subgroups. Depression detection within CI subjects yields lower overall performance, with mild improvements from low and mid-level W2V2 layers. Cross-task generalization between CI and depression classification is limited, indicating that each task depends on distinct representations. These findings emphasize the need for fairness-aware model evaluation and subgroup-specific analysis in clinical speech applications, particularly in light of demographic and clinical heterogeneity in real-world applications.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This study systematically investigates bias in self-supervised acoustic representations for cognitive impairment detection, revealing significant performance disparities across demographic and clinical subgroups. The comprehensive methodology and rigorous experimental evaluation contribute valuable insights into the fairness and reliability of machine learning models in clinical applications.
Methodology
The methodology is robust, employing a systematic bias analysis of acoustic representations for cognitive impairment detection. The comparison of traditional acoustic features with self-supervised embeddings from Wav2Vec 2.0 is well-structured, and the evaluation across demographic and clinical subgroups adds significant depth to the analysis. The use of multiple classifiers and detailed bias metrics enhances the rigor of the methodology.
Results & evaluation
The experiments are comprehensive, utilizing a well-defined dataset (DementiaBank Pitt Corpus) and addressing class imbalance through various balancing strategies. The results are clearly presented, demonstrating the performance of different acoustic features and classifiers, with a focus on subgroup-specific metrics. However, the performance for depression detection is notably lower, which raises questions about the model's effectiveness in this area.
Reproducibility
The paper provides a GitHub repository with source code and audio filenames, which supports reproducibility. However, the raw audio recordings cannot be shared due to restrictions, which may limit the ability of other researchers to fully replicate the study.
Limitations
The study is limited by its reliance on a single dataset, which may not capture the full demographic and linguistic diversity of clinical populations. Additionally, the small number of depression-labeled samples in the dataset may affect the robustness of the conclusions regarding depression classification.
Broader impact
The findings highlight critical issues related to bias and fairness in machine learning applications in healthcare, particularly in speech-based diagnostics for cognitive impairment. The work underscores the importance of fairness-aware evaluation protocols, which could influence future research and clinical practices in AI-driven healthcare solutions. This study systematically investigates bias in self-supervised acoustic representations for cognitive impairment detection, revealing significant performance disparities across demographic and clinical subgroups. The comprehensive methodology and rigorous experimental evaluation contribute valuable insights into the fairness and reliability of machine learning models in clinical applications.
Author of this paper? Send a correction.