Aliasing-Free Neural Audio Synthesis
Abstract
Neural vocoders and codecs reconstruct waveforms from acoustic representations, which directly impact the audio quality. Among existing methods, upsampling-based time-domain models are superior in both inference speed and synthesis quality, achieving state-of-the-art performance. Still, despite their success in producing perceptually natural sound, their synthesis fidelity remains limited due to the aliasing artifacts brought by the inadequately designed model architectures. In particular, the unconstrained nonlinear activation generates an infinite number of harmonics that exceed the Nyquist frequency, resulting in ``folded-back'' aliasing artifacts. The widely used upsampling layer, ConvTranspose, copies the mirrored low-frequency parts to fill the empty high-frequency region, resulting in ``mirrored'' aliasing artifacts. Meanwhile, the combination of its inherent periodicity and the mirrored DC bias also brings ``tonal artifact,'' resulting in constant-frequency ringing. This paper aims to solve these issues from a signal processing perspective. Specifically, we apply oversampling and anti-derivative anti-aliasing to the activation function to obtain its anti-aliased form, and replace the problematic ConvTranspose layer with resampling to avoid the ``tonal artifact'' and eliminate aliased components. Based on our proposed anti-aliased modules, we introduce Pupu-Vocoder and Pupu-Codec, and release high-quality pre-trained checkpoints to facilitate audio generation research. We build a test signal benchmark to illustrate the effectiveness of the anti-aliased modules, and conduct experiments on speech, singing voice, music, and audio to validate our proposed models. Experimental results confirm that our lightweight Pupu-Vocoder and Pupu-Codec models can easily outperform existing systems on singing voice, music, and audio, while achieving comparable performance on speech.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of anti-aliased activation functions and upsampling methods that significantly improve the fidelity of neural audio synthesis models. This work represents a meaningful advancement in the field of audio generation, bridging the gap between signal processing and deep learning to tackle a well-known challenge in audio synthesis.
Methodology
The paper introduces a novel approach to address aliasing artifacts in neural audio synthesis by applying anti-derivative anti-aliasing (ADAA) techniques to activation functions and replacing the ConvTranspose layer with a resampling method. This approach is grounded in signal processing principles, which adds a layer of theoretical robustness to the proposed models, Pupu-Vocoder and Pupu-Codec. The methodology is well-structured and clearly articulated, showcasing a thoughtful integration of signal processing concepts into deep learning architectures.
Results & evaluation
The experimental setup is comprehensive, utilizing a diverse set of datasets across multiple domains (speech, singing voice, music, and audio). The results demonstrate the superiority of the proposed models over existing systems, with detailed metrics provided for both objective and subjective evaluations. The use of a test signal benchmark to validate the effectiveness of the anti-aliased modules is a strong point, as it provides clear evidence of the improvements made.
Reproducibility
The paper includes sufficient implementation details, including model architectures, training configurations, and evaluation metrics, which facilitates reproducibility. However, the absence of a public code repository limits the ease of access for researchers wishing to replicate the results.
Limitations
One limitation is the reliance on specific datasets, which may not fully represent the diversity of audio synthesis challenges. Additionally, while the proposed models outperform existing systems, the paper does not address potential trade-offs in computational efficiency or real-time performance, which are critical in practical applications.
Broader impact
The proposed methods have significant implications for audio synthesis applications, including speech synthesis, music generation, and sound design. By effectively mitigating aliasing artifacts, the research can enhance audio quality in various domains, potentially influencing both academic research and industry practices. The main contribution of this paper is the introduction of anti-aliased activation functions and upsampling methods that significantly improve the fidelity of neural audio synthesis models. This work represents a meaningful advancement in the field of audio generation, bridging the gap between signal processing and deep learning to tackle a well-known challenge in audio synthesis.
Author of this paper? Send a correction.