Toward Interpretable Speech Deepfake Detection using Artifact-Specific Experts and Calibrated Detection Scores
Abstract
In this work, we propose an interpretable framework for speech deepfake detection based on artifact-specific expert models. Rather than relying on black-box decisions, the framework provides human-understandable evidence, which is critical in high-stakes settings. Each expert is trained to detect a specific speech synthesis artifact, and its output is calibrated into a log-likelihood ratio that serves as an interpretable evidence score. We evaluate five artifact-specific experts and show that, with proper calibration, they can capture their target artifacts and produce meaningful evidence. Importantly, each expert estimates only the presence of its assigned artifact rather than directly performing the final decision. Their outputs are aggregated into an ensemble to produce the actual real-versus-fake classification, while maintaining interpretability by indicating how strongly each expert supports or contradicts a fake classification. Results show that artifact-specific experts capture interpretable signals of synthetic speech across multiple generation pipelines.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a novel, interpretable framework for speech deepfake detection by decomposing the problem into artifact-specific experts trained on pseudo-fake data and aggregating their calibrated log-likelihood ratios, offering a transparent alternative to black-box models in forensic contexts.
Methodology
The paper proposes a modular, interpretable framework for speech deepfake detection that decomposes the detection task into five specific synthesis artifacts: F0 inconsistencies, oversmoothing, degraded phase, energy inconsistencies, and tonal artifacts. The core methodological innovation lies in the use of "pseudo-fake" data generation—applying controlled, algorithmic manipulations to real speech to train artifact-specific expert models. This avoids the confounding factors present in real deepfake data. The outputs of these experts are calibrated into Log-Likelihood Ratios (LLRs) using either Kernel Density Estimation (KDE) or Constrained Maximum-Likelihood Gaussian (CMLG) methods, allowing for a principled, forensic-style aggregation of evidence. The approach is grounded in signal processing principles rather than purely end-to-end deep learning, aiming for intrinsic interpretability.
Results & evaluation
The authors evaluate their framework on ASVspoof 2019, ASVspoof 5, and SpoofCeleb datasets. They demonstrate that individual experts can detect their target artifacts with above-chance performance, though not state-of-the-art accuracy. The ensemble of calibrated experts achieves competitive results, with the sum-aggregation strategy outperforming max and learned gating mechanisms. The interpretability is validated by showing that experts activate strongly for attacks known to exhibit those specific artifacts (e.g., the phase expert for Neural Source-Filter vocoders). The paper also includes an ablation study on calibration data composition, finding that a pooled calibration set across experts yields the most stable LLRs.
Reproducibility
The paper provides detailed descriptions of the pseudo-fake generation pipelines, including specific parameters for manipulations (e.g., Gaussian filter standard deviation, F0 scaling factors). The feature extraction methods (pYIN, LSF, STFT) and model architectures (MLPs, LCNN) are specified. However, the exact code for the pseudo-fake generation and the specific random seeds for the experiments are not provided in the text, which may hinder exact reproduction. The reliance on external libraries (e.g., pYIN) is noted but implementation details are standard.
Limitations
The primary limitation is the reliance on predefined artifacts. The framework cannot detect novel or unknown artifacts that do not fit the five modeled categories. The performance is lower than state-of-the-art black-box detectors, which is acknowledged but remains a practical constraint. The assumption of independence among artifact features for LLR summation is a simplification that may not hold perfectly in complex synthesis pipelines. Furthermore, the "pseudo-fake" approach may not fully capture the complexity and distribution of artifacts found in real-world, multi-stage deepfake generation systems.
Broader impact
This work contributes to the critical field of audio forensics and security by providing a transparent method for deepfake detection. The interpretable nature of the system makes it suitable for high-stakes applications where explaining the decision to a human judge or jury is required. It shifts the paradigm from opaque black-box detectors to evidence-based reasoning, potentially increasing trust in automated detection systems. This paper presents a novel, interpretable framework for speech deepfake detection by decomposing the problem into artifact-specific experts trained on pseudo-fake data and aggregating their calibrated log-likelihood ratios, offering a transparent alternative to black-box models in forensic contexts.
Author of this paper? Send a correction.