Towards Array-Invariant Speech Enhancement via Geometry-Aware Dynamic Convolution
Abstract
Multi-channel speech enhancement (SE) systems exhibit superior performance over single-channel methods but are constrained to fixed microphone array configurations. This restricts their real-world deployment across devices with diverse array geometries. While recent array-agnostic SE methods address variable microphone numbers and permutations, they largely fail to exploit explicit array geometry priors when available, missing a crucial cue for optimal spatial filtering. A Geometry-Aware Dynamic Convolution (Geo-DConv) framework is proposed, which explicitly leverages microphone coordinates to transform standard fixed-array SE models into robust array-invariant systems. Experiments are conducted on the recent real-recorded RealMAN multi-channel speech dataset. Results demonstrate that the proposed architecture enables two widely used fixed-array models to adapt to array-invariant settings, with consistent performance improvements across diverse array topologies.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a practical and effective method for adapting fixed-array speech enhancement models to arbitrary microphone geometries using explicit geometric priors via dynamic convolution, demonstrating improved generalization on real-world data.
Methodology
The paper proposes a Geometry-Aware Dynamic Convolution (Geo-DConv) module to enable fixed-array speech enhancement models to handle arbitrary microphone array geometries. The core innovation lies in using a Topology-Aware Coordinate Transformer (TACT) to generate dynamic convolution weights based on relative microphone coordinates. This approach attempts to bridge the gap between fixed-array models (which exploit geometry implicitly through training) and array-agnostic models (which often ignore explicit geometry). The method is theoretically sound, leveraging Fourier positional encoding and self-attention to ensure permutation equivariance. However, the novelty is moderate as dynamic convolution and coordinate-based conditioning are established techniques in other domains (e.g., medical imaging, computer vision), and their application here is an incremental adaptation rather than a fundamental algorithmic breakthrough.
Results & evaluation
The experiments are conducted on the RealMAN dataset, which is a significant strength as it uses real-recorded data, addressing the common sim-to-real gap in multi-channel SE. The authors compare their method against strong baselines (FaSNet-TAC, USES2-comp, SpatialNet, TF-GridNet). The results show that Geo-DConv improves upon existing array-agnostic methods and allows fixed-array models to generalize to unseen array configurations. The evaluation includes metrics like SDR, SI-SDR, PESQ, STOI, and DNSMOS. The generalization to the CHiME-4 dataset (6 mics) is a positive finding. However, the performance gains, while consistent, are not massive, and the computational overhead of the TACT module is not extensively analyzed in terms of inference latency compared to the baseline fixed-array models.
Reproducibility
The paper provides detailed architectural descriptions, including hyperparameters for the TACT module (hidden dimension, number of heads, layers) and the Geo-DConv layer (basis dimension, output channels). The dataset (RealMAN) is publicly available, and the implementation details (STFT parameters, sampling rate) are clear. The code is not explicitly linked, but the mathematical formulations are sufficient for reproduction by a competent researcher. The use of standard libraries for STFT and common neural network components further aids reproducibility.
Limitations
The primary limitation is the assumption that microphone coordinates are known or can be accurately estimated. In real-world scenarios with unknown or dynamically changing arrays, this prior might not be available. Additionally, the method relies on a fixed basis dimension for dynamic convolution, which might limit the capacity to model highly complex or large-scale array geometries compared to methods that process raw channel interactions directly. The performance on very small arrays (1-2 mics) is still constrained by the fundamental limits of single-channel or stereo enhancement. The paper does not extensively discuss the sensitivity of the TACT module to coordinate normalization or scaling.
Broader impact
This work contributes to the broader goal of robust, device-agnostic audio processing. By enabling fixed-array models to adapt to various geometries, it reduces the need for device-specific training and deployment, facilitating the integration of advanced SE into diverse consumer devices (smartphones, laptops, IoT devices) with varying hardware configurations. It highlights the importance of explicit geometric priors in multi-channel audio tasks. The paper presents a practical and effective method for adapting fixed-array speech enhancement models to arbitrary microphone geometries using explicit geometric priors via dynamic convolution, demonstrating improved generalization on real-world data.
Author of this paper? Send a correction.