BigVGAN: A Universal Neural Vocoder with Large-Scale Training
Abstract
Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous speakers across various recording environments. In this work, we present BigVGAN, a universal vocoder that generalizes well for various out-of-distribution scenarios without fine-tuning. We introduce periodic activation function and anti-aliased representation into the GAN generator, which brings the desired inductive bias for audio synthesis and significantly improves audio quality. In addition, we train our GAN vocoder at the largest scale up to 112M parameters, which is unprecedented in the literature. We identify and address the failure modes in large-scale GAN training for audio, while maintaining high-fidelity output without over-regularization. Our BigVGAN, trained only on clean speech (LibriTTS), achieves the state-of-the-art performance for various zero-shot (out-of-distribution) conditions, including unseen speakers, languages, recording environments, singing voices, music, and instrumental audio. We release our code and model at: https://github.com/NVIDIA/BigVGAN
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of BigVGAN, a universal neural vocoder that leverages large-scale training and innovative architectural components to achieve state-of-the-art performance in high-fidelity audio synthesis across various out-of-distribution scenarios. The comprehensive evaluation and significant improvements over existing models highlight its potential impact on the field of audio synthesis and generative models.
Methodology
The methodology presented in BigVGAN is robust, introducing novel architectural components such as the periodic activation function and anti-aliased representation, which are shown to significantly enhance the model's performance in generating high-fidelity audio across diverse conditions. The paper effectively addresses the challenges of large-scale GAN training, providing empirical insights into the stability of training and the importance of model capacity.
Results & evaluation
The experimental evaluation is comprehensive, utilizing a diverse dataset (LibriTTS) and conducting both objective and subjective assessments of audio quality. The results demonstrate significant improvements over baseline models, particularly in zero-shot scenarios, which is a critical aspect for practical applications in voice synthesis and audio generation.
Reproducibility
The authors have made their code and models publicly available, which is a positive aspect for reproducibility. However, the paper could benefit from more detailed descriptions of hyperparameters and training configurations to ensure that other researchers can replicate the results effectively.
Limitations
One limitation is the reliance on a single dataset (LibriTTS) for training, which may affect the generalizability of the model to other datasets or real-world scenarios. Additionally, while the paper addresses training stability, the potential for early collapse in large models remains a concern that could impact usability in practical applications.
Broader impact
The advancements made in this paper have significant implications for various applications, including voice cloning, speech synthesis, and audio coding. The ability to generate high-fidelity audio across diverse conditions without fine-tuning opens up new possibilities for real-time applications in multimedia and communication technologies. The main contribution of this paper is the introduction of BigVGAN, a universal neural vocoder that leverages large-scale training and innovative architectural components to achieve state-of-the-art performance in high-fidelity audio synthesis across various out-of-distribution scenarios. The comprehensive evaluation and significant improvements over existing models highlight its potential impact on the field of audio synthesis and generative models.
Curator note
Lee et al., NVIDIA; scaled HiFi-GAN with anti-aliased activations; strong universal vocoder
Author of this paper? Send a correction.