DeepGESI: A Non-Intrusive Objective Evaluation Model for Predicting Speech Intelligibility in Hearing-Impaired Listeners
Abstract
Speech intelligibility assessment is essential for many speech-related applications. However, most objective intelligibility metrics are intrusive, as they require clean reference speech in addition to the degraded or processed signal for evaluation. Furthermore, existing metrics such as STOI are primarily designed for normal hearing listeners, and their predictive accuracy for hearing impaired speech intelligibility remains limited. On the other hand, the GESI (Gammachirp Envelope Similarity Index) can be used to estimate intelligibility for hearing-impaired listeners, but it is also intrusive, as it depends on reference signals. This requirement limits its applicability in real-world scenarios. To overcome this limitation, this study proposes DeepGESI, a non-intrusive deep learning-based model capable of accurately and efficiently predicting the speech intelligibility of hearing-impaired listeners without requiring any clean reference speech. Experimental results demonstrate that, under the test conditions of the 2nd Clarity Prediction Challenge(CPC2) dataset, the GESI scores predicted by DeepGESI exhibit a strong correlation with the actual GESI scores. In addition, the proposed model achieves a substantially faster prediction speed compared to conventional methods.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of DeepGESI, a non-intrusive deep learning model for predicting speech intelligibility in hearing-impaired listeners, which significantly advances the field by providing a practical solution to a longstanding challenge in speech assessment. The technical contributions, particularly in methodology and experimental validation, position this work as a valuable addition to the literature on speech intelligibility metrics.
Methodology
The proposed DeepGESI model is innovative in its non-intrusive approach to predicting speech intelligibility for hearing-impaired listeners. The architecture effectively combines acoustic feature extraction through STFT and learnable filterbanks, along with an attention mechanism to capture salient information. The use of Maxout activation functions and Rotary Position Embedding for positional encoding demonstrates a thoughtful adaptation of existing techniques to improve performance in speech intelligibility tasks. However, while the methodology is sound, it heavily relies on the underlying assumptions of the GESI framework, which may limit its generalizability.
Results & evaluation
The experiments leverage the CPC2 dataset, which is appropriate for the task and allows for robust evaluation against both seen and unseen datasets. The reported metrics (MSE, LCC, SRCC) indicate strong performance, particularly in terms of correlation with ground truth values. The paper provides sufficient detail on the experimental setup, including the computational resources used, which enhances the credibility of the results. However, the absence of comparisons with a broader range of existing non-intrusive metrics limits the contextual understanding of DeepGESI's performance.
Reproducibility
The paper outlines the architecture and training process in sufficient detail, allowing for potential reproduction of the results. However, the lack of publicly available code or a demo URL is a significant drawback, as it hinders other researchers from validating the findings independently. Including a GitHub repository or similar resource would greatly enhance reproducibility.
Limitations
The study acknowledges that DeepGESI was not fine-tuned using subjective listening tests, which is a critical aspect for validating intelligibility predictions. Additionally, while the model shows strong performance on the CPC2 dataset, its effectiveness in more diverse or real-world acoustic conditions remains untested. The reliance on a single dataset may also raise concerns about overfitting and generalizability.
Broader impact
The implications of this research are significant, particularly for the development of hearing aids and cochlear implants, as it addresses a critical gap in non-intrusive intelligibility assessment for hearing-impaired individuals. By enabling real-time processing and reducing the need for clean reference speech, DeepGESI could facilitate advancements in speech technology applications, improving accessibility for hearing-impaired listeners in various environments. The main contribution of this paper is the introduction of DeepGESI, a non-intrusive deep learning model for predicting speech intelligibility in hearing-impaired listeners, which significantly advances the field by providing a practical solution to a longstanding challenge in speech assessment. The technical contributions, particularly in methodology and experimental validation, position this work as a valuable addition to the literature on speech intelligibility metrics.
Author of this paper? Send a correction.