DuraMark: Duration-Embedded Watermarking in LLM-based TTS
Abstract
Large language model (LLM)-based text-to-speech (TTS) models have achieved remarkable voice cloning capabilities, raising concerns about potential deepfake misuse. Speech watermarking mitigates this by embedding traceable information into generated speech. Mainstream watermarking methods operate at the signal level (waveform or spectrogram), rendering the watermark vulnerable to generative attacks (e.g., neural codec and vocoder). To address this, we propose DuraMark, a robust information-level watermarking framework. It utilizes syllable duration editing to achieve watermark embedding. Specifically, DuraMark integrates a duration-controllable LLM-based TTS model to edit syllable durations during synthesis, coupled with a duration extractor to extract these durations for detection. Experiments demonstrate DuraMark's superior robustness against generative attacks, significantly outperforming signal-level baselines. Audio samples are available at https://muzw.github.io/duramark_demo/.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of DuraMark, a novel generative watermarking framework that embeds watermarks into synthesized speech by editing syllable durations, significantly improving robustness against generative attacks while preserving speech quality. This work represents a meaningful advancement in the field of audio processing and watermarking, addressing critical concerns related to deepfake technologies and the integrity of synthesized speech.
Methodology
The proposed DuraMark framework introduces a novel approach to watermarking in LLM-based TTS systems by embedding watermarks at the information level through syllable duration editing. This method is innovative as it leverages a duration-controllable TTS model and a duration extractor, which allows for precise control over the watermarking process while maintaining the naturalness of the synthesized speech. The integration of these components is well-structured, and the methodology is clearly articulated, allowing for a thorough understanding of the process.
Results & evaluation
The experiments conducted are robust, utilizing a substantial dataset and comparing DuraMark against established signal-level watermarking methods. The evaluation metrics include True Positive Rate (TPR) under various attack scenarios, which is a relevant measure of robustness. The results demonstrate DuraMark's superior performance, particularly against generative attacks, which is a critical aspect of the paper's claims. The use of both objective and subjective metrics to assess speech naturalness further strengthens the experimental evaluation.
Reproducibility
The paper provides sufficient detail regarding the experimental setup, including the datasets used and the training parameters. However, the absence of a public code repository limits reproducibility. While the methodology is clearly described, access to the code would enhance the ability of other researchers to validate and build upon this work.
Limitations
One limitation is the reliance on a specific language (Mandarin Chinese) for the experiments, which may affect the generalizability of the findings to other languages or dialects. Additionally, while the paper demonstrates robustness against various attacks, it does not explore the performance of DuraMark under more extreme or novel attack scenarios that may arise in real-world applications.
Broader impact
The implications of this research are significant, particularly in the context of combating deepfake technologies and ensuring the integrity of synthesized speech. The DuraMark framework could be applied in various fields, including media, security, and digital forensics, where the authenticity of audio content is crucial. The potential for this technology to enhance trust in AI-generated content is noteworthy. The main contribution of this paper is the introduction of DuraMark, a novel generative watermarking framework that embeds watermarks into synthesized speech by editing syllable durations, significantly improving robustness against generative attacks while preserving speech quality. This work represents a meaningful advancement in the field of audio processing and watermarking, addressing critical concerns related to deepfake technologies and the integrity of synthesized speech.
Author of this paper? Send a correction.