Can Current Detectors Catch Face-to-Voice Deepfake Attacks?
Abstract
The rapid advancement of generative models has enabled the creation of increasingly stealthy synthetic voices, commonly referred to as audio deepfakes. A recent technique, FOICE [USENIX'24], demonstrates a particularly alarming capability: generating a victim's voice from a single facial image, without requiring any voice sample. By exploiting correlations between facial and vocal features, FOICE produces synthetic voices realistic enough to bypass industry-standard authentication systems, including WeChat Voiceprint and Microsoft Azure. This raises serious security concerns, as facial images are far easier for adversaries to obtain than voice samples, dramatically lowering the barrier to large-scale attacks. In this work, we investigate two core research questions: (RQ1) can state-of-the-art audio deepfake detectors reliably detect FOICE-generated speech under clean and noisy conditions, and (RQ2) whether fine-tuning these detectors on FOICE data improves detection without overfitting, thereby preserving robustness to unseen voice generators such as SpeechT5. Our study makes three contributions. First, we present the first systematic evaluation of FOICE detection, showing that leading detectors consistently fail under both standard and noisy conditions. Second, we introduce targeted fine-tuning strategies that capture FOICE-specific artifacts, yielding significant accuracy improvements. Third, we assess generalization after fine-tuning, revealing trade-offs between specialization to FOICE and robustness to unseen synthesis pipelines. These findings expose fundamental weaknesses in today's defenses and motivate new architectures and training protocols for next-generation audio deepfake detection.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a significant advancement in understanding the vulnerabilities of audio deepfake detection systems against novel face-to-voice synthesis attacks. The comprehensive evaluation and innovative fine-tuning strategies provide valuable insights into the limitations of current defenses and the urgent need for more robust detection methodologies.
Methodology
The methodology is robust, featuring a systematic evaluation framework that includes diverse detection models and a well-constructed dataset that reflects real-world conditions. The fine-tuning strategies are clearly articulated, addressing both adaptation to FOICE-specific cues and generalization to unseen synthesis methods. The inclusion of various noise conditions adds depth to the evaluation.
Results & evaluation
The experiments are thorough, with a clear focus on both in-distribution and out-of-distribution performance. The results are well-presented, showcasing the significant performance improvements achieved through fine-tuning while also highlighting the trade-offs involved. The use of multiple metrics (EER, accuracy, precision, recall, F1-score) provides a comprehensive view of detector performance.
Reproducibility
The paper emphasizes reproducibility by using publicly available models and clearly detailing the training and evaluation protocols. However, the lack of a project URL or demo limits the ease of access to the implementation for external researchers.
Limitations
The study does not explore the performance of FOICE combined with alternative vocoders or the impact of real-world playback conditions extensively. Additionally, the reliance on fine-tuning as a reactive measure may not be sufficient for long-term defense strategies.
Broader impact
The findings have significant implications for the security of voice authentication systems, highlighting vulnerabilities that could be exploited in real-world scenarios. The research underscores the need for adaptive defenses in the face of evolving deepfake technologies. This paper presents a significant advancement in understanding the vulnerabilities of audio deepfake detection systems against novel face-to-voice synthesis attacks. The comprehensive evaluation and innovative fine-tuning strategies provide valuable insights into the limitations of current defenses and the urgent need for more robust detection methodologies.
Author of this paper? Send a correction.