When Entanglement Lower-Bounds Disparity: Auditing and Repairing Demographic Fairness in Audio Understanding Models
Abstract
Speech technology penalizes some voices: recognition errs nearly twice as often for Black speakers, and accuracy declines for second-language accents and older speakers. We introduce TRIAD, an audit grid crossing 120 texts, 24 rendered demographic voice profiles (gender, age band, accent), and ten expressive styles via controllable text-to-speech, isolating perceived demographic attributes from content and affect. For ten open-weights encoders we define axis-fidelity functionals, principal-angle leakage between axis subspaces, and group-conditional gaps; a proposition proves that average probe disparity grows with the same aggregate voice-semantic leakage $Λ$ we measure, and a corollary shows that peak leakage forces worst-case disparity inside an active region. The measured mean-square probe disparity tracks $Λ$ (Pearson r = 0.93), and a black-box protocol exposes the same signature in two closed-source models. ORCA, an adapter combining axis-specific contrastive heads, an orthogonality penalty, and group-balanced sampling, cuts leakage 72% and roughly halves the gaps.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a geometric framework linking subspace entanglement to demographic disparity in speech encoders, validated by a large-scale synthetic audit grid and a lightweight repair adapter. It makes a significant contribution to the theory of fairness in representation learning by providing provable bounds on disparity based on measurable geometric properties, although its reliance on synthetic data for primary validation limits the strength of its causal claims.
Methodology
The paper proposes a rigorous geometric framework for auditing demographic fairness in speech encoders. The core contribution is the definition of "axis-fidelity" and "subspace leakage" (principal angles between semantic and voice subspaces). The theoretical contribution is a proposition linking aggregate leakage to average probe disparity and a corollary linking peak leakage to worst-case disparity. This is a strong methodological advance in moving fairness from black-box output metrics to white-box representation geometry. The proposed repair mechanism, ORCA, uses a residual adapter with orthogonal contrastive heads, which is a sound architectural choice for disentangling factors without destroying utility.
Results & evaluation
The experimental setup is extensive, utilizing a custom "TRIAD" grid of 28,800 synthetic utterances to control for confounding variables (content, affect, demographics). The evaluation covers 10 open-weights encoders and 2 closed-source models. The correlation between measured leakage and disparity (r=0.93) is a compelling empirical validation of the theory. The ablation studies effectively isolate the contribution of the orthogonality penalty. However, the reliance on a single TTS system for the grid introduces a significant validity threat, which the authors acknowledge but only partially mitigate with real-speech validation checks.
Reproducibility
The paper claims that the grid, estimators, and code are available in the Supplementary Material. However, no specific URLs or repository links are provided in the text. The detailed hyperparameters (AdamW, lr=1e-3, 20k steps) and dataset construction details (Gemini 3.1 Flash TTS) are provided, which aids reproducibility if the supplementary materials are accessible. The use of a specific, potentially proprietary or rapidly changing TTS model (Gemini 3.1) may limit long-term reproducibility.
Limitations
The primary limitation is the reliance on synthetic data for the main audit grid. While the authors perform validation on real speech (CANDOR, SSSD, etc.), the causal attribution of disparity to demographic attributes is weaker in synthetic data due to potential TTS artifacts. The theory is limited to linear probes; non-linear disparities are not bounded by the proposed leakage metrics. The institution "People Make Things" is obscure, raising questions about the resources and peer-review rigor compared to major academic or industrial labs.
Broader impact
This work provides a principled tool for diagnosing bias in speech models, which is critical as these models are deployed in high-stakes applications (healthcare, legal, customer service). The geometric perspective offers a new lens for understanding why certain models are biased, moving beyond correlation to structural entanglement. The ORCA adapter offers a practical, lightweight fix that can be applied to existing frozen models. The paper introduces a geometric framework linking subspace entanglement to demographic disparity in speech encoders, validated by a large-scale synthetic audit grid and a lightweight repair adapter. It makes a significant contribution to the theory of fairness in representation learning by providing provable bounds on disparity based on measurable geometric properties, although its reliance on synthetic data for primary validation limits the strength of its causal claims.
Author of this paper? Send a correction.