S-PRESSO: Ultra Low Bitrate Sound Effect Compression With Diffusion Autoencoders And Offline Quantization
Abstract
Neural audio compression models have recently achieved extreme compression rates, enabling efficient latent generative modeling. Conversely, latent generative models have been applied to compression, pushing the limits of continuous and discrete approaches. However, existing methods remain constrained to low-resolution audio and degrade substantially at very low bitrates, where audible artifacts are prominent. In this paper, we present S-PRESSO, a 48kHz sound effect compression model that produces both continuous and discrete embeddings at ultra-low bitrates, down to 0.096 kbps, via offline quantization. Our model relies on a pretrained latent diffusion model to decode compressed audio embeddings learned by a latent encoder. Leveraging the generative priors of the diffusion decoder, we achieve extremely low frame rates, down to 1Hz (750x compression rate), producing convincing and realistic reconstructions at the cost of exact fidelity. Despite operating at high compression rates, we demonstrate that S-PRESSO outperforms both continuous and discrete baselines in audio quality, acoustic similarity and reconstruction metrics.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents S-PRESSO, a diffusion autoencoder for ultra-low bitrate audio compression, achieving significant improvements in audio quality while maintaining high compression rates. This work highlights the potential of generative models to redefine audio compression standards, pushing the boundaries of what is achievable in the field.
Methodology
The paper introduces S-PRESSO, a novel approach to audio compression utilizing a diffusion autoencoder framework. The methodology is well-structured, comprising a three-step training process that includes continuous diffusion autoencoder training, offline quantization, and diffusion decoder finetuning. This approach effectively leverages the generative capabilities of diffusion models to enhance audio quality at ultra-low bitrates. The use of pretrained models for both the latent encoder and the diffusion decoder is a strong point, as it allows for the incorporation of learned representations that can significantly improve the compression process. However, the paper could benefit from a more detailed explanation of the quantization process and its impact on the overall performance.
Results & evaluation
The experimental setup is robust, utilizing a diverse set of datasets that cover various audio types, which enhances the generalizability of the results. The authors provide a thorough comparison against both continuous and discrete baseline models, demonstrating significant improvements in audio quality metrics such as FAD, KAD, and Si-SDR. The subjective evaluation through MUSHRA tests adds credibility to the findings, although the paper does not discuss the statistical significance of the results in detail. Overall, the experiments convincingly support the claims made about the performance of S-PRESSO.
Reproducibility
The paper includes sufficient implementation details, including training parameters and architecture specifications, which aids in reproducibility. However, the absence of publicly available code or models limits the ability of other researchers to replicate the results fully. The authors mention the use of specific datasets but do not provide access to these datasets, which could hinder reproducibility for others in the field.
Limitations
One notable limitation is the focus on sound effects, which may restrict the applicability of the proposed method to other audio domains such as music or speech. Additionally, while the results are promising, the trade-off between compression rate and audio fidelity could be further explored, particularly at the lowest bitrates. The paper also acknowledges the need for improvements in inference speed, which is crucial for practical applications.
Broader impact
The advancements in ultra-low bitrate audio compression have significant implications for various applications, including gaming, virtual reality, and streaming services, where bandwidth is a critical concern. By shifting the focus from strict fidelity to acoustic similarity, this work opens new avenues for audio representation and synthesis, potentially enhancing user experiences in interactive media. The findings could also inspire further research into generative models for audio processing. The paper presents S-PRESSO, a diffusion autoencoder for ultra-low bitrate audio compression, achieving significant improvements in audio quality while maintaining high compression rates. This work highlights the potential of generative models to redefine audio compression standards, pushing the boundaries of what is achievable in the field.
Author of this paper? Send a correction.