VeRe-Flow: Guiding Flow Matching toward Clean Speech via Velocity Contrastive Regularization and Representation Alignment for Noise-Robust Bandwidth Expansion
Abstract
Noise-robust bandwidth expansion aims to reconstruct high-fidelity wideband speech from noisy low-resolution inputs. While flow matching has shown strong performance in speech generation, accurately recovering clean speech from noisy inputs remains challenging due to the ambiguity of velocity estimation under noise. In this work, we propose VeRe-Flow, a clean-guided flow matching framework that introduces multi-level clean supervision to guide the generative process toward clean speech. At the velocity level, we introduce velocity contrastive regularization, which attracts the predicted velocity toward the clean trajectory while repelling it from noisy trajectories. At the representation level, we incorporate representation alignment that aligns intermediate features with clean self-supervised learning representations. The results demonstrate that the proposed method achieves the lowest LSD and highest DNSMOS OVRL among all baselines, and the highest MOS among generative baselines.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents VeRe-Flow, a flow matching framework for noise-robust bandwidth expansion that introduces velocity contrastive regularization and representation alignment to guide the generative process toward clean speech manifolds. While the methodological novelty is incremental compared to the broader landscape of generative audio, the empirical results demonstrate a clear improvement in objective and subjective metrics, making it a solid contribution to the specific subfield of speech enhancement and bandwidth expansion.
Methodology
The paper proposes VeRe-Flow, a flow matching framework for noise-robust bandwidth expansion (NR-BWE). The core technical contributions are two regularization terms: Velocity Contrastive Regularization (VeCoR) and Representation Alignment. VeCoR attempts to guide the velocity field by attracting it toward clean trajectories and repelling it from noisy ones. Representation Alignment uses a projection head to align intermediate transformer features with clean self-supervised learning (SSL) embeddings (specifically from XEUS). The architecture combines Convolutional ResBlocks and Transformer blocks, conditioned on noisy low-resolution mel-spectrograms and SSL features. While the integration of SSL features is established in recent speech literature, the specific application of contrastive regularization on the velocity field of a flow matching model for this specific task is a novel methodological contribution. However, the theoretical grounding for why velocity contrastive learning is superior to standard conditional flow matching or diffusion-based noise modeling in this specific context is not deeply explored mathematically.
Results & evaluation
The experiments are conducted on the Valentini-Botinhao dataset, a standard benchmark for NR-BWE. The authors compare against generative baselines (FLowHigh, NU-Wave2) and non-generative methods. They report objective metrics (LSD, DNSMOS) and subjective metrics (MOS). The results indicate that VeRe-Flow outperforms baselines in LSD and DNSMOS OVRL. The ablation studies provide insight into the contribution of each component (Conv ResBlocks, XEUS, REPA, VeCoR). The evaluation is thorough for the scope of the paper, covering both spectral fidelity and perceptual quality. The use of DNSMOS is appropriate for speech enhancement tasks. However, the comparison with non-generative baselines is limited to reported numbers from other papers, which may introduce inconsistencies in evaluation protocols (e.g., vocoder differences, though BigVGAN is used for the proposed method and FLowHigh).
Reproducibility
The paper provides sufficient implementation details, including dataset preprocessing (Chebyshev filter parameters), model architecture (Conv ResBlock structure, transformer depth), training hyperparameters (optimizer, learning rate, batch size, loss weights), and the specific SSL model used (XEUS). The use of publicly available components (BigVGAN, XEUS, Valentini-Botinhao) enhances reproducibility. The code is not explicitly linked in the text provided (only a demo URL), which is a minor drawback for immediate reproducibility, but the description is detailed enough for a competent researcher to implement.
Limitations
The paper does not discuss the computational cost or inference speed of VeRe-Flow compared to baselines. Flow matching models can be sensitive to the choice of ODE solvers and number of function evaluations (NFE); while they mention testing different settings, the optimal trade-off between quality and speed is not analyzed. The reliance on SSL features (XEUS) introduces a dependency on an external model, which might not be available or compatible with all deployment scenarios. Furthermore, the "repulsion" term in VeCoR requires careful tuning of the temperature or margin parameter; the paper reports a fixed weight but does not discuss the sensitivity of this hyperparameter. The claim of being the "first to apply velocity contrastive regularization to speech generation" is strong and should be verified against recent diffusion-based contrastive works.
Broader impact
This work contributes to the field of speech processing by improving the quality of bandwidth expansion in noisy environments, which has applications in telecommunications, hearing aids, and audio restoration. By leveraging flow matching, it offers a potentially faster alternative to diffusion models for high-quality speech generation. The integration of SSL representations highlights the trend of using self-supervised features to guide generative processes, which can be generalized to other audio tasks. The paper presents VeRe-Flow, a flow matching framework for noise-robust bandwidth expansion that introduces velocity contrastive regularization and representation alignment to guide the generative process toward clean speech manifolds. While the methodological novelty is incremental compared to the broader landscape of generative audio, the empirical results demonstrate a clear improvement in objective and subjective metrics, making it a solid contribution to the specific subfield of speech enhancement and bandwidth expansion.
Author of this paper? Send a correction.