DPDFNet: Boosting DeepFilterNet2 via Dual-Path RNN
Abstract
We present DPDFNet, a causal single-channel speech enhancement model that extends DeepFilterNet2 architecture with dual-path blocks in the encoder, strengthening long-range temporal and cross-band modeling while preserving the original enhancement framework. In addition, we demonstrate that adding a loss component to mitigate over-attenuation in the enhanced speech, combined with a fine-tuning phase tailored for "always-on" applications, leads to substantial improvements in overall model performance. To compare our proposed architecture with a variety of causal open-source models, we created a new evaluation set comprising long, low-SNR recordings in 12 languages across everyday noise scenarios, better reflecting real-world conditions than commonly used benchmarks. On this evaluation set, DPDFNet delivers superior performance to other causal open-source models, including some that are substantially larger and more computationally demanding. We also propose an holistic metric named PRISM, a composite, scale-normalized aggregate of intrusive and non-intrusive metrics, which demonstrates clear scalability with the number of dual-path blocks. We further demonstrate on-device feasibility by deploying DPDFNet on Ceva-NeuPro-Nano edge NPUs. Results indicate that DPDFNet-4, our second-largest model, achieves real-time performance on NPN32 and runs even faster on NPN64, confirming that state-of-the-art quality can be sustained within strict embedded power and latency constraints.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the development of DPDFNet, a novel speech enhancement model that effectively integrates dual-path recurrent neural networks into the existing DeepFilterNet2 architecture, achieving superior performance in real-world noisy environments. This work significantly advances the state-of-the-art in single-channel speech enhancement by addressing critical challenges in real-time processing and model robustness.
Methodology
The methodology presented in DPDFNet is robust, extending the DeepFilterNet2 architecture by integrating dual-path recurrent neural networks (DPRNN) to enhance long-range temporal and cross-band modeling. The addition of an over-attenuation loss function and a fine-tuning phase specifically designed for "always-on" applications demonstrates a thoughtful approach to addressing common challenges in speech enhancement. The paper also introduces a new evaluation set that better reflects real-world conditions, which is a significant improvement over existing benchmarks.
Results & evaluation
The experimental evaluation is comprehensive, utilizing a new multilingual low-SNR dataset that includes recordings in 12 languages and various everyday noise scenarios. The results indicate that DPDFNet consistently outperforms existing models, including larger and more computationally demanding alternatives. The introduction of the PRISM metric provides a holistic view of model performance, integrating both intrusive and non-intrusive measures effectively.
Reproducibility
The paper provides sufficient implementation details, including training protocols, dataset descriptions, and evaluation metrics, which facilitate reproducibility. The availability of code and pretrained models on GitHub further supports this aspect, allowing other researchers to replicate the findings.
Limitations
One limitation is that while the paper addresses over-attenuation, it does not explore the potential trade-offs between computational efficiency and enhancement quality in depth. Additionally, the real-time performance of the largest model (DPDFNet-8) does not meet real-time constraints, which may limit its applicability in certain scenarios.
Broader impact
The advancements made in DPDFNet have significant implications for real-time speech enhancement applications, particularly in mobile and embedded systems where computational resources are limited. The ability to deploy high-quality models on edge devices could enhance user experiences in various applications, such as telecommunication, virtual assistants, and hearing aids. The main contribution of this paper is the development of DPDFNet, a novel speech enhancement model that effectively integrates dual-path recurrent neural networks into the existing DeepFilterNet2 architecture, achieving superior performance in real-world noisy environments. This work significantly advances the state-of-the-art in single-channel speech enhancement by addressing critical challenges in real-time processing and model robustness.
Author of this paper? Send a correction.