VoCodec: A Low-bitrate Streamable Neural Speech Codec with Voicing-driven Quantization
Abstract
Neural speech codecs are key to speech transmission and storage, but most use uniform quantization across frames, allocating the same bitrate regardless of content and wasting bits. We propose VoCodec, a low-bitrate streamable neural speech codec with voicing-driven quantization that assigns higher bitrate to voiced frames and lower bitrate to unvoiced frames according to perceptual sensitivity. VoCodec embeds a voicing detector in a fully causal encoder-quantizer-decoder neural coding framework, using residual scalar-vector quantization for voiced frames and simple scalar quantization for unvoiced ones. Experiments show that on the LibriTTS dataset at a 16 kHz sampling rate, VoCodec outperforms baseline neural speech codecs even at a bitrate as low as 1.1 kbps. Our further experiments also confirm that introducing voicing-driven quantization can effectively reduce the bitrate by approximately 27% compared with uniform quantization strategy.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
VoCodec presents a novel approach to low-bitrate neural speech coding through its voicing-driven quantization strategy. This work represents a meaningful advancement in the field of speech processing, combining innovative methodology with rigorous experimental validation to address key challenges in speech transmission and storage.
Methodology
The methodology presented in VoCodec is innovative, particularly in its voicing-driven quantization approach that dynamically allocates bitrates based on the voiced/unvoiced characteristics of speech. The integration of a voicing detector within a fully causal encoder-quantizer-decoder framework is a significant advancement over traditional uniform quantization methods. The use of residual scalar-vector quantization for voiced frames and simpler scalar quantization for unvoiced frames is well-justified, leveraging perceptual sensitivity to optimize bitrate usage effectively. The architecture is detailed and demonstrates a thoughtful design that balances complexity and performance.
Results & evaluation
The experimental evaluation is robust, utilizing both objective metrics (LSD, STOI, ViSQOL) and subjective assessments (MUSHRA, ABX tests) to validate the performance of VoCodec against several baseline neural speech codecs. The choice of datasets (LibriTTS and VCTK) is appropriate for the task, and the results indicate that VoCodec achieves significant improvements in coding quality at lower bitrates. The paper provides clear comparisons with baseline models, demonstrating the effectiveness of the proposed method. However, further exploration of different datasets could strengthen the generalizability of the findings.
Reproducibility
The paper includes sufficient details about the architecture, training parameters, and evaluation metrics, which aids in reproducibility. However, the absence of a publicly available code repository limits the ease with which other researchers can replicate the results. Providing a GitHub repository or similar would enhance the reproducibility of the work.
Limitations
One limitation is the focus on specific datasets, which may not fully represent the diversity of speech patterns encountered in real-world applications. Additionally, while the paper claims significant bitrate savings, the impact of this on various types of speech (e.g., different accents or languages) is not explored. The reliance on subjective evaluations also introduces variability that could affect the perceived quality of the codec.
Broader impact
The proposed VoCodec has the potential to significantly impact applications in low-bitrate speech transmission, such as in mobile communications, voice-over-IP services, and streaming applications where bandwidth is limited. Its ability to maintain high perceptual quality while reducing bitrate could lead to more efficient use of network resources and improved user experiences in real-time communication scenarios. The methodology could also inspire further research in audio processing and compression techniques. VoCodec presents a novel approach to low-bitrate neural speech coding through its voicing-driven quantization strategy. This work represents a meaningful advancement in the field of speech processing, combining innovative methodology with rigorous experimental validation to address key challenges in speech transmission and storage.
Author of this paper? Send a correction.