Do Multimodal Large Language Models Need Reasoning to Classify Dementia from Speech?
Abstract
Multimodal large language models (MLLMs) have emerged as a promising approach for improving the accuracy, transferability, and explainability of automatic dementia classification (ADC) systems from voice recordings. Yet it remains unclear whether their reasoning capabilities are beneficial for ADC, and how such capabilities should be leveraged. In this paper, we conduct a careful evaluation of reasoning MLLMs for ADC and show that naive strategies, such as relying on text-based rationales, can lead to hallucinated and inconsistent rationales for diagnosis and yield inferior ADC performance compared with LLM-free baselines. To overcome this limitation, we propose \textbf{De}mentia \textbf{T}hinker with Nonlinear \textbf{A}daptor and Re\textbf{i}nforcement \textbf{L}earning (DeTAiL), an adaptor-based framework that exploits the internal representations of reasoning MLLMs for improved dementia classification. Across two dementia datasets with distinct test formats and label granularities, DeTAiL consistently outperforms strong baselines and methods that rely on text-based rationales. Code and demo will be released upon acceptance.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a rigorous evaluation of reasoning MLLMs for dementia classification, proposing DeTAiL to leverage internal representations for improved accuracy and transferability, offering valuable insights into the utility of reasoning traces in medical speech analysis.
Methodology
The paper proposes a novel framework, DeTAiL, to investigate whether reasoning capabilities in Multimodal Large Language Models (MLLMs) are beneficial for Automatic Dementia Classification (ADC). The core methodological contribution is the "nonlinear adaptor" stage, which extracts hidden representations from the MLLM conditioned on generated rationales, rather than relying solely on the textual output. This is combined with a distillation stage (using a teacher LLM to generate rationales) and a Reinforcement Learning stage (GRPO) to align the model. The approach attempts to bridge the gap between the generative reasoning capabilities of LLMs and the discriminative needs of speech classification, addressing the issue of hallucinated or unfaithful rationales by probing internal states. The methodology is sound and addresses a relevant gap in understanding how reasoning traces interact with downstream classification tasks in medical speech analysis.
Results & evaluation
The evaluation is conducted on two datasets: ADReSS (binary classification) and LEADS (fine-grained classification). The experiments cover in-domain performance, cross-domain transfer, and ablation studies on input modalities and layer selection. The results demonstrate that while naive reasoning strategies (text-based) can underperform or hallucinate, the proposed DeTAiL framework consistently outperforms strong baselines, including LoRA-based adaptation and text-only MLLM approaches. The cross-domain analysis is particularly valuable, showing that the hidden-state adaptor offers better transferability than LoRA in some settings. However, the paper notes that the MLP adaptor can overfit to dataset-specific patterns, which is a critical finding. The inclusion of a reliability analysis of linguistic evidence types adds depth to the evaluation.
Reproducibility
The paper provides detailed descriptions of the datasets, model architectures (Qwen-2.5-VL), and training hyperparameters (LoRA rank, GRPO group size, learning rates). The use of open-source models and standard toolkits (ms-swift) enhances reproducibility. The authors state that code and demo will be released upon acceptance, which is standard for arXiv submissions. The specific details regarding the MLP adaptor structure and the distillation process are sufficiently described to allow replication.
Limitations
The paper acknowledges several limitations. First, the LEADS dataset is private, which limits independent verification and broader community benchmarking. Second, the ASR transcripts used for LEADS may introduce noise, affecting the quality of rationales and hidden states. Third, the cross-domain transfer performance is not uniform; while DeTAiL helps, it does not fully solve the domain shift problem. Fourth, the reliability of the generated rationales is still an open question, as the paper suggests they may not always be faithful clinical explanations. Finally, the study is limited to specific MLLMs (Qwen family) and datasets, so generalizability to other models or languages is not fully established.
Broader impact
This work has significant implications for the development of explainable and robust AI systems for healthcare, specifically in neurodegenerative disease screening. By demonstrating that internal representations of reasoning MLLMs can be more reliable than their textual outputs for classification, it provides a pathway for building systems that are both accurate and interpretable. However, the caution regarding hallucinated rationales highlights the need for rigorous validation in clinical settings. The findings contribute to the broader understanding of how to effectively leverage large models for specialized, high-stakes tasks where reasoning is often assumed to be beneficial but may not be directly transferable. The paper presents a rigorous evaluation of reasoning MLLMs for dementia classification, proposing DeTAiL to leverage internal representations for improved accuracy and transferability, offering valuable insights into the utility of reasoning traces in medical speech analysis.
Author of this paper? Send a correction.