Lightweight Resolution-Aware Audio Deepfake Detection via Cross-Scale Attention and Consistency Learning
Abstract
Audio deepfake detection has become increasingly challenging due to rapid advances in speech synthesis and voice conversion technologies, particularly under channel distortions, replay attacks, and real-world recording conditions. This paper proposes a resolution-aware audio deepfake detection framework that explicitly models and aligns multi-resolution spectral representations through cross-scale attention and consistency learning. Unlike conventional single-resolution or implicit feature-fusion approaches, the proposed method enforces agreement across complementary time--frequency scales. The proposed framework is evaluated on three representative benchmarks: ASVspoof 2019 (LA and PA), the Fake-or-Real (FoR) dataset, and the In-the-Wild Audio Deepfake dataset under a speaker-disjoint protocol. The method achieves near-perfect performance on ASVspoof LA (EER 0.16%), strong robustness on ASVspoof PA (EER 5.09%), FoR rerecorded audio (EER 4.54%), and in-the-wild deepfakes (AUC 0.98, EER 4.81%), significantly outperforming single-resolution and non-attention baselines under challenging conditions. The proposed model remains lightweight and efficient, requiring only 159k parameters and less than 1~GFLOP per inference, making it suitable for practical deployment. Comprehensive ablation studies confirm the critical contributions of cross-scale attention and consistency learning, while gradient-based interpretability analysis reveals that the model learns resolution-consistent and semantically meaningful spectral cues across diverse spoofing conditions. These results demonstrate that explicit cross-resolution modeling provides a principled, robust, and scalable foundation for next-generation audio deepfake detection systems.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a resolution-aware audio deepfake detection framework that significantly enhances detection capabilities by explicitly modeling interactions among multiple spectral resolutions. The comprehensive evaluation and innovative methodology position this work as a valuable contribution to the field of audio processing and machine learning.
Methodology
The proposed methodology introduces a resolution-aware framework that leverages cross-scale attention and consistency learning to enhance audio deepfake detection. This approach is innovative as it explicitly models interactions among multiple spectral resolutions, addressing limitations of existing methods that typically rely on single-resolution features. The architecture is well-structured, with a shared encoder and attention mechanism that adaptively integrates information across resolutions. The use of a consistency learning objective to enforce resolution-invariant characteristics is particularly noteworthy, as it aims to improve robustness against channel distortions and replay attacks.
Results & evaluation
The experimental evaluation is comprehensive, utilizing three diverse datasets: ASVspoof 2019, Fake-or-Real (FoR), and In-the-Wild Audio Deepfake datasets. The results demonstrate strong performance, achieving near-perfect accuracy on controlled benchmarks and maintaining robustness under challenging conditions. The ablation studies effectively highlight the contributions of different components of the model, confirming the importance of cross-scale attention and consistency learning in enhancing detection capabilities.
Reproducibility
The paper mentions that the code will be made publicly available upon acceptance, which is a positive aspect for reproducibility. However, the lack of a demo URL or direct access to the code at this stage limits immediate reproducibility. The methodology is described in sufficient detail, allowing for potential replication of the experiments.
Limitations
Some limitations include the exclusive reliance on Mel-spectrogram representations, which may not capture all relevant features for audio deepfake detection. Additionally, the consistency learning objective is applied only to bona fide speech, which may not universally apply across all recording conditions. The paper also acknowledges the evolving nature of generative speech models, suggesting that the proposed method may need continuous adaptation to remain effective.
Broader impact
The proposed framework has significant implications for security and forensic applications, as it addresses the growing threat of audio deepfakes. By providing a robust detection mechanism that generalizes well across various conditions, this work can contribute to the development of reliable systems for identifying manipulated audio, thereby enhancing trust in audio communications and media. This paper presents a resolution-aware audio deepfake detection framework that significantly enhances detection capabilities by explicitly modeling interactions among multiple spectral resolutions. The comprehensive evaluation and innovative methodology position this work as a valuable contribution to the field of audio processing and machine learning.
Author of this paper? Send a correction.