Measuring Soft Biometric Leakage in Speaker De-Identification Systems
Abstract
We use the term re-identification to refer to the process of recovering the original speaker's identity from anonymized speech outputs. Speaker de-identification systems aim to reduce the risk of re-identification, but most evaluations focus only on individual-level measures and overlook broader risks from soft biometric leakage. We introduce the Soft Biometric Leakage Score (SBLS), a unified method that quantifies resistance to zero-shot inference attacks on non-unique traits such as channel type, age range, dialect, sex of the speaker, or speaking style. SBLS integrates three elements: direct attribute inference using pre-trained classifiers, linkage detection via mutual information analysis, and subgroup robustness across intersecting attributes. Applying SBLS with publicly available classifiers, we show that all five evaluated de-identification systems exhibit significant vulnerabilities. Our results indicate that adversaries using only pre-trained models - without access to original speech or system details - can still reliably recover soft biometric information from anonymized output, exposing fundamental weaknesses that standard distributional metrics fail to capture.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper makes a significant contribution by introducing a novel metric for assessing soft biometric leakage in speaker de-identification systems, revealing vulnerabilities that traditional metrics overlook. The comprehensive methodology and experimental evaluation underscore the importance of addressing privacy concerns in speech processing, although further work is needed to enhance reproducibility and generalizability.
Methodology
The paper introduces the Soft Biometric Leakage Score (SBLS), a novel metric that integrates three components: zero-shot attribute inference, systematic linkage detection, and subgroup robustness. This comprehensive approach addresses a significant gap in the evaluation of speaker de-identification systems by focusing on soft biometric leakage rather than traditional metrics. The methodology is well-structured and employs established statistical techniques, such as mutual information analysis, to quantify vulnerabilities effectively. However, the choice of heuristic weights in the SBLS calculation could benefit from further justification or exploration of alternative weighting strategies.
Results & evaluation
The experimental setup is robust, utilizing a diverse dataset (Mixer 3 corpus) with rich demographic annotations to evaluate five different speaker de-identification systems. The results demonstrate significant vulnerabilities in all evaluated systems, highlighting the effectiveness of the SBLS in revealing soft biometric leakage. The paper provides clear performance metrics, including AUC values and subgroup-specific leakage scores, which enhance the interpretability of the findings. However, the reliance on a single dataset may limit the generalizability of the results.
Reproducibility
While the paper outlines the methodology and experimental setup in detail, it lacks specific implementation details or code availability, which could hinder reproducibility. The absence of a public repository for the SBLS implementation or the evaluated systems means that other researchers cannot easily replicate the experiments or build upon the findings.
Limitations
The primary limitation is the focus on a single dataset, which may not capture the full spectrum of speaker characteristics and de-identification challenges. Additionally, the paper does not address potential variations in performance across different languages or dialects, which could affect the applicability of the findings. The heuristic nature of the SBLS component weights also raises questions about the robustness of the results.
Broader impact
This research has significant implications for privacy and security in speech processing applications, particularly in contexts where speaker anonymity is crucial. By quantifying soft biometric leakage, the findings can inform the development of more robust speaker de-identification systems and contribute to the broader discourse on privacy-preserving technologies in machine learning. The introduction of SBLS could lead to improved standards for evaluating de-identification systems, ultimately enhancing user trust and safety in voice-based applications. The paper makes a significant contribution by introducing a novel metric for assessing soft biometric leakage in speaker de-identification systems, revealing vulnerabilities that traditional metrics overlook. The comprehensive methodology and experimental evaluation underscore the importance of addressing privacy concerns in speech processing, although further work is needed to enhance reproducibility and generalizability.
Author of this paper? Send a correction.