A multimodal Bayesian Network for symptom-level depression and anxiety prediction from voice and speech data
Abstract
During psychiatric assessment, clinicians observe not only what patients report, but important nonverbal signs such as tone, speech rate, fluency, responsiveness, and body language. Weighing and integrating these different information sources is a challenging task and a good candidate for support by intelligence-driven tools - however this is yet to be realized in the clinic. Here, we argue that several important barriers to adoption can be addressed using Bayesian network modelling. To demonstrate this, we evaluate a model for depression and anxiety symptom prediction from voice and speech features in large-scale datasets (30,135 unique speakers). Alongside performance for conditions and symptoms (for depression, anxiety ROC-AUC=0.842,0.831 ECE=0.018,0.015; core individual symptom ROC-AUC>0.74), we assess demographic fairness and investigate integration across and redundancy between different input modality types. Clinical usefulness metrics and acceptability to mental health service users are explored. When provided with sufficiently rich and large-scale multimodal data streams and specified to represent common mental conditions at the symptom rather than disorder level, such models are a principled approach for building robust assessment support tools: providing clinically-relevant outputs in a transparent and explainable format that is directly amenable to expert clinical supervision.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the development of a multimodal Bayesian network model that predicts depression and anxiety symptoms from voice and speech data, offering a promising approach to enhance psychiatric assessments through the integration of diverse data sources. The methodology is innovative, the experimental evaluation is thorough, and the implications for clinical practice are substantial, positioning this work as a significant advancement in the field of mental health technology.
Methodology
The paper presents a novel approach using Bayesian networks to integrate multimodal data (voice and speech features) for predicting symptoms of depression and anxiety. The methodology is well-structured, leveraging a large dataset of over 30,000 unique speakers to ensure robustness and generalizability. The use of surrogate models to compress high-dimensional feature sets into manageable representations for the Bayesian network is innovative and demonstrates a thoughtful approach to handling complex data. The model's ability to provide symptom-level predictions rather than binary classifications is a significant advancement in the field of psychiatric assessment, allowing for more nuanced clinical insights.
Results & evaluation
The experiments are comprehensive, utilizing a large and diverse dataset for training, calibration, and testing. The reported performance metrics (ROC-AUC values) indicate strong discrimination capabilities for both overall conditions and individual symptoms, with calibration metrics supporting the model's clinical utility. The inclusion of fairness assessments across demographic groups adds depth to the evaluation, addressing a critical concern in AI applications in healthcare. However, the paper could benefit from clearer visualizations of results and more explicit comparisons with existing models.
Reproducibility
The methodology is described in sufficient detail to allow for reproducibility, including the data collection process, model architecture, and training procedures. However, the lack of a publicly available code repository or project URL limits the ease of reproduction for external researchers. Future work should consider making the model and data available for verification and further research.
Limitations
The study acknowledges limitations, including reliance on self-reported data for symptom assessment, which may introduce bias. Additionally, the model's performance across diverse accents and non-native speakers requires further validation before real-world implementation. There is also a need for ongoing user acceptability testing to ensure the tool meets the needs of both clinicians and patients.
Broader impact
The potential applications of this research are significant, particularly in enhancing psychiatric assessments and supporting clinicians in decision-making processes. By providing a tool that integrates nonverbal cues with traditional assessments, the model could improve diagnostic accuracy and patient outcomes. The focus on demographic fairness also positions this work as a meaningful contribution to addressing equity in mental health care. The main contribution of this paper is the development of a multimodal Bayesian network model that predicts depression and anxiety symptoms from voice and speech data, offering a promising approach to enhance psychiatric assessments through the integration of diverse data sources. The methodology is innovative, the experimental evaluation is thorough, and the implications for clinical practice are substantial, positioning this work as a significant advancement in the field of mental health technology.
Author of this paper? Send a correction.