MeanFlowSE: One-Step Generative Speech Enhancement via MeanFlow
Abstract
Speech enhancement (SE) recovers clean speech from noisy signals and is vital for applications such as telecommunications and automatic speech recognition (ASR). While generative approaches achieve strong perceptual quality, they often rely on multi-step sampling (diffusion/flow-matching) or large language models, limiting real-time deployment. To mitigate these constraints, we present MeanFlowSE, a one-step generative SE framework. It adopts MeanFlow to predict an average-velocity field for one-step latent refinement and conditions the model on self-supervised learning (SSL) representations rather than VAE latents. This design accelerates inference and provides robust acoustic-semantic guidance during training. In the Interspeech 2020 DNS Challenge blind test set and simulated test set, MeanFlowSE attains state-of-the-art (SOTA) level perceptual quality and competitive intelligibility while significantly lowering both real-time factor (RTF) and model size compared with recent generative competitors, making it suitable for practical use. The code will be released upon publication at https://github.com/Hello3orld/MeanFlowSE.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a novel one-step generative speech enhancement model, MeanFlowSE, which integrates self-supervised learning for improved performance and efficiency. This contribution is significant as it addresses the computational challenges of existing generative models, paving the way for practical applications in real-time speech processing.
Methodology
The proposed MeanFlowSE framework innovatively integrates a one-step generative model with self-supervised learning (SSL) representations, which enhances the quality of speech enhancement while significantly reducing computational demands. The use of MeanFlow to predict an average-velocity field for latent refinement is a novel approach that bypasses the need for multi-step sampling, thus improving real-time applicability. The architecture is modular, allowing for efficient training and inference, which is a strong point of the methodology. However, the paper could benefit from a more detailed discussion on the choice of SSL models and their impact on performance.
Results & evaluation
The experiments are well-structured, utilizing the Interspeech 2020 DNS Challenge dataset to evaluate the performance of MeanFlowSE against state-of-the-art models. The results show that MeanFlowSE achieves competitive scores in terms of perceptual quality and intelligibility while maintaining low real-time factors and model size. The inclusion of ablation studies provides valuable insights into the contributions of different components of the model, reinforcing the effectiveness of SSL conditioning over VAE latents.
Reproducibility
The paper provides sufficient implementation details, including model configurations, training protocols, and hyperparameters, which are crucial for reproducibility. The authors have committed to releasing the code upon publication, which is a positive aspect for the community. However, further details on the training environment and any specific dependencies would enhance reproducibility.
Limitations
One limitation is the reliance on SSL models, which may not be universally applicable across all speech enhancement scenarios. Additionally, while the paper demonstrates significant improvements in real-time factors, the specific computational requirements and trade-offs in various deployment environments are not thoroughly discussed. The impact of different noise types and conditions on performance could also be explored more comprehensively.
Broader impact
The advancements in speech enhancement through MeanFlowSE have significant implications for real-time applications in telecommunications, hearing aids, and automatic speech recognition. By improving the efficiency and effectiveness of speech enhancement models, this work can contribute to better communication technologies, particularly in challenging acoustic environments. The potential for deployment on low-resource devices makes this research particularly relevant in the context of increasing demand for accessible communication tools. The paper presents a novel one-step generative speech enhancement model, MeanFlowSE, which integrates self-supervised learning for improved performance and efficiency. This contribution is significant as it addresses the computational challenges of existing generative models, paving the way for practical applications in real-time speech processing.
Author of this paper? Send a correction.