Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech
Abstract
We propose a unified framework for not only attributing synthetic speech to its source but also for detecting speech generated by synthesizers that were not encountered during training. This requires methods that move beyond simple detection to support both detailed forensic analysis and open-set generalization. To address this, we introduce SIGNAL, a hybrid framework that combines speech foundation models (SFMs) with graph-based modeling and open-set-aware inference. Our framework integrates Graph Neural Networks (GNNs) and a k-Nearest Neighbor (KNN) classifier, allowing it to capture meaningful relationships between utterances and recognize speech that doesn`t belong to any known generator. It constructs a query-conditioned graph over generator class prototypes, enabling the GNN to reason over relationships among candidate generators, while the KNN branch supports open-set detection via confidence-based thresholding. We evaluate SIGNAL using the DiffSSD dataset, which offers a diverse mix of real speech and synthetic audio from both open-source and commercial diffusion-based TTS systems. To further assess generalization, we also test on the SingFake benchmark. Our results show that SIGNAL consistently improves performance across both tasks, with Mamba-based embeddings delivering especially strong results. To the best of our knowledge, this is the first study to unify graph-based learning and open-set detection for tracing synthetic speech back to its origin.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents SIGNAL, a hybrid framework that effectively combines GNNs and KNN for robust synthetic speech detection and source attribution. This innovative approach has the potential to significantly advance the field of audio forensics by improving the detection of synthetic speech and addressing the challenges posed by unseen generators.
Methodology
The proposed SIGNAL framework integrates Graph Neural Networks (GNNs) and k-Nearest Neighbors (KNN) to tackle the dual challenges of source attribution and open-set detection in synthetic speech. The methodology is well-structured, leveraging GNNs for relational modeling and KNN for instance-based reasoning. The construction of a query-conditioned graph over generator class prototypes is a novel approach that enhances the model's ability to reason over relationships among candidate generators. The combination of these two methodologies is innovative, although the paper could benefit from a more detailed explanation of the underlying assumptions and potential biases in the graph construction process.
Results & evaluation
The experiments are comprehensive, utilizing two diverse datasets (DiffSSD and SingFake) to evaluate the framework's performance in both closed-set and open-set scenarios. The results demonstrate consistent improvements across various tasks, particularly with Mamba-based embeddings. However, the paper could enhance its credibility by providing more detailed statistical analyses and comparisons with existing state-of-the-art methods.
Reproducibility
The paper provides a GitHub repository link for the SIGNAL framework, which is a positive aspect for reproducibility. However, the implementation details, such as hyperparameter settings and training procedures, could be more explicitly outlined to facilitate replication by other researchers.
Limitations
The authors acknowledge several limitations, including the reliance on a fixed decision threshold for open-set detection and the lack of fine-grained attribution among multiple unseen sources. Additionally, the evaluation is limited to two benchmarks, which may not encompass all possible synthesis conditions or generators.
Broader impact
The proposed framework addresses critical issues in synthetic speech detection, which has significant implications for security, misinformation, and trust in AI systems. By improving the ability to trace synthetic speech back to its source, this work contributes to the broader field of digital forensics and responsible AI deployment. The paper presents SIGNAL, a hybrid framework that effectively combines GNNs and KNN for robust synthetic speech detection and source attribution. This innovative approach has the potential to significantly advance the field of audio forensics by improving the detection of synthetic speech and addressing the challenges posed by unseen generators.
Author of this paper? Send a correction.