AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos
Abstract
Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a high-quality, large-scale benchmark (AV-FABLE) and a novel method (NFSR) for audio-visual manipulation forensics in long-form videos, demonstrating that frozen AV-LLM features are more effective for forensic localization than zero-shot language outputs.
Methodology
The paper introduces AV-FABLE, a substantial benchmark for sparse audio-visual manipulations in long-form videos, featuring a dual-timeline evaluation protocol that distinguishes audio and visual tracks. This is a significant methodological advancement over joint-timeline evaluations which can mask modality-specific errors. The proposed model, NFSR, utilizes frozen features from AV-LLMs (Qwen3-Omni) with separate temporal decoders for each modality. The key methodological insight is the observation that while zero-shot prompting of AV-LLMs fails at forensic detection, their frozen perceptual features contain strong forensic signals accessible via supervised readouts. The use of host-conditioned feature modulation (FiLM) is a clever architectural choice to contextualize local anomalies within the broader video content.
Results & evaluation
The experimental setup is rigorous, including in-domain and three out-of-distribution (OOD) settings (source transfer, held-out generators, held-out routes). The dataset scale (16,650 pairs, 998 hours) is impressive for a forensic benchmark. Results show NFSR outperforming strong baselines like UMMAFormer and ActionFormer in AV-mAP across all settings. The ablation studies effectively demonstrate the contribution of intermediate-layer features, host conditioning, and modality separation. The comparison with zero-shot LLMs highlights a critical gap in current multimodal LLM capabilities for fine-grained forensic tasks.
Reproducibility
The authors commit to releasing the dataset, code, and model weights. The paper provides detailed descriptions of the data construction pipeline (SEE-Forge), including the use of Gemini 3.7 Flash for planning and human review protocols for quality control. The evaluation metrics (AV-mAP, Event-IoU) and protocols are clearly defined. The reliance on specific, potentially proprietary, LLMs for data generation (Gemini) might pose minor reproducibility challenges for the dataset creation itself, but the benchmark and model are designed to be open.
Limitations
The benchmark relies on synthetic manipulations generated by specific AI models, which may not fully capture the diversity of real-world human-edited forgeries or future, unseen generation techniques. The "explanation" task is noted as challenging and not fully solved by current baselines. The paper acknowledges that visual coverage under distribution shift remains an open problem, with NFSR showing lower V-IoU compared to some visual-only baselines in certain OOD settings.
Broader impact
This work addresses a critical gap in AI safety and media integrity by providing tools to localize and attribute manipulations in long-form content, which is increasingly relevant for misinformation detection. The finding that AV-LLMs encode forensic evidence in their features but fail to expose it via language outputs is a valuable insight for the broader multimodal AI community, suggesting that current alignment or decoding strategies in LLMs may be suboptimal for fine-grained perception tasks. The paper presents a high-quality, large-scale benchmark (AV-FABLE) and a novel method (NFSR) for audio-visual manipulation forensics in long-form videos, demonstrating that frozen AV-LLM features are more effective for forensic localization than zero-shot language outputs.
Author of this paper? Send a correction.