Towards Blind Data Cleaning: A Case Study in Music Source Separation
Abstract
The performance of deep learning models for music source separation heavily depends on training data quality. However, datasets are often corrupted by difficult-to-detect artifacts such as audio bleeding and label noise. Since the type and extent of contamination are typically unknown, cleaning methods targeting specific corruptions are often impractical. This paper proposes and evaluates two distinct, noise-agnostic data cleaning methods to address this challenge. The first approach uses data attribution via unlearning to identify and filter out training samples that contribute the least to producing clean outputs. The second leverages the Fr\'echet Audio Distance to measure and remove samples that are perceptually dissimilar to a small and trusted clean reference set. On a dataset contaminated with a simulated distribution of real-world noise, our unlearning-based methods produced a cleaned dataset and a corresponding model that outperforms both the original contaminated data and the small clean reference set used for cleaning. This result closes approximately 66.7\% of the performance gap between the contaminated baseline and a model trained on the same dataset without any contamination. Unlike methods tailored for specific artifacts, our noise-agnostic approaches offer a more generic and broadly applicable solution for curating high-quality training data.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a significant advancement in blind data cleaning for music source separation, proposing two innovative methods that enhance model performance by effectively addressing data quality issues. The comprehensive evaluation of these methods demonstrates their potential impact on the field and beyond.
Methodology
The paper introduces two innovative noise-agnostic data cleaning methods for music source separation: an unlearning-based approach and a distributional metric-based approach using Fréchet Audio Distance. The unlearning method is particularly noteworthy as it flips the traditional influence estimation, focusing on the impact of clean samples to filter out noisy ones. This methodological shift is a significant contribution to the field, offering a more generalized solution to data cleaning that can be applied across various domains. The use of distributional metrics to assess perceptual dissimilarity adds robustness to the approach, allowing for effective cleaning without prior knowledge of specific corruptions.
Results & evaluation
The experiments are well-structured, utilizing a semi-synthetic dataset to simulate real-world noise and contamination. The results demonstrate that the proposed methods outperform both the contaminated baseline and a small clean reference set, effectively closing the performance gap with a model trained on entirely clean data. The comparative analysis with baseline methods, including a classifier-based approach, highlights the advantages of the proposed noise-agnostic methods in terms of generalizability and robustness.
Reproducibility
The paper provides sufficient detail on the methodologies and experimental setup, including the model architecture, training processes, and evaluation metrics. However, the lack of specific URLs for code or datasets limits the ease of reproducibility. Future work could benefit from making the code and datasets publicly available to enhance reproducibility.
Limitations
The study acknowledges limitations in determining the optimal filtering ratio, which requires retraining and could be automated in future work. Additionally, the methods were only tested on a specific model architecture (Open-Unmix), which may limit their applicability to more complex models. The performance on unseen audio effects also indicates that while the methods are robust, they may not generalize perfectly across all types of noise.
Broader impact
The proposed methods have significant implications for improving data quality in machine learning applications beyond music source separation, such as speech enhancement and sound event detection. By providing a more generic solution to data cleaning, this research could facilitate the development of more robust models in various audio processing tasks. The paper presents a significant advancement in blind data cleaning for music source separation, proposing two innovative methods that enhance model performance by effectively addressing data quality issues. The comprehensive evaluation of these methods demonstrates their potential impact on the field and beyond.
Author of this paper? Send a correction.