DeePAQ: A Perceptual Audio Quality Metric Based On Foundational Models and Weakly Supervised Learning
Abstract
This paper presents the Deep learning-based Perceptual Audio Quality metric (DeePAQ) for evaluating general audio quality. Our approach leverages metric learning together with the music foundation model MERT, guided by surrogate labels, to construct an embedding space that captures distortion intensity in general audio. To the best of our knowledge, DeePAQ is the first in the general audio quality domain to leverage weakly supervised labels and metric learning for fine-tuning a music foundation model with Low-Rank Adaptation (LoRA), a direction not yet explored by other state-of-the-art methods. We benchmark the proposed model against state-of-the-art objective audio quality metrics across listening tests spanning audio coding and source separation. Results show that our method surpasses existing metrics in detecting coding artifacts and generalizes well to unseen distortions such as source separation, highlighting its robustness and versatility.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
DeePAQ introduces a novel perceptual audio quality metric that leverages weakly supervised learning and metric learning to adapt a music foundation model for audio quality assessment. The innovative methodology, robust experimental validation, and potential applications highlight its significance in advancing the field of audio quality evaluation.
Methodology
The methodology presented in DeePAQ is innovative, leveraging weakly supervised learning and metric learning to adapt a music foundation model (MERT) for perceptual audio quality assessment. The use of Low-Rank Adaptation (LoRA) to fine-tune the model is particularly noteworthy, as it allows for effective adaptation with a limited number of parameters. The approach of using surrogate labels for training is a clever solution to the scarcity of subjective ratings in the audio domain, although the potential noise introduced by these labels is acknowledged. The RnC loss function is a significant contribution that enables the model to learn a quality-related embedding space effectively. However, the paper could benefit from a more thorough exploration of the limitations of the surrogate labels and their impact on the model's performance.
Results & evaluation
The experimental evaluation is robust, with a comprehensive benchmarking against state-of-the-art audio quality metrics across multiple listening tests. The results demonstrate that DeePAQ consistently outperforms existing metrics, particularly in detecting coding artifacts and generalizing to unseen distortions. The use of diverse datasets and listening tests enhances the credibility of the findings. However, the paper could provide more detailed statistical analysis of the results to further substantiate the claims of superiority over existing methods.
Reproducibility
The paper provides a detailed description of the training setup, including the dataset composition, training parameters, and evaluation metrics, which aids in reproducibility. However, the lack of a publicly available implementation or code repository limits the ability for others to replicate the results independently. Including a link to a GitHub repository or similar would significantly enhance reproducibility.
Limitations
The primary limitation identified is the reliance on surrogate labels, which may introduce noise and affect the model's performance. Additionally, while the model shows strong performance in audio coding tasks, its effectiveness in source separation is less pronounced, indicating potential areas for improvement. The paper also does not address the scalability of the approach to larger datasets or different audio contexts.
Broader impact
The development of DeePAQ has significant implications for the field of audio quality assessment, particularly in applications where subjective evaluation is impractical. The ability to effectively evaluate audio quality using a computational metric can enhance various domains, including music streaming, audio coding, and telecommunications. The approach could also pave the way for further research into the use of foundation models in other audio-related tasks. DeePAQ introduces a novel perceptual audio quality metric that leverages weakly supervised learning and metric learning to adapt a music foundation model for audio quality assessment. The innovative methodology, robust experimental validation, and potential applications highlight its significance in advancing the field of audio quality evaluation.
Author of this paper? Send a correction.