TsetlinKWS: A 65nm 16.58uW, 0.63mm2 State-Driven Convolutional Tsetlin Machine-Based Accelerator For Keyword Spotting
Abstract
The Tsetlin Machine (TM) has recently attracted attention as a low-power alternative to neural networks due to its simple and interpretable inference mechanisms. However, its performance on speech-related tasks remains limited. This paper proposes TsetlinKWS, the first algorithm-hardware co-design framework for the Convolutional Tsetlin Machine (CTM) on the 12-keyword spotting task. Firstly, we introduce a novel Mel-Frequency Spectral Coefficient and Spectral Flux (MFSC-SF) feature extraction scheme together with spectral convolution, enabling the CTM to reach its first-ever competitive accuracy of 87.35% on the 12-keyword spotting task. Secondly, we develop an Optimized Grouped Block-Compressed Sparse Row (OG-BCSR) algorithm that achieves a remarkable 9.84$\times$ reduction in model size, significantly improving the storage efficiency on CTMs. Finally, we propose a state-driven architecture tailored for the CTM, which simultaneously exploits data reuse and sparsity to achieve high energy efficiency. The full system is evaluated in 65 nm process technology, consuming 16.58 $\mu$W at 0.7 V with a compact 0.63 mm$^2$ core area. TsetlinKWS requires only 907k logic operations per inference, representing a 10$\times$ reduction compared to the state-of-the-art KWS accelerators, positioning the CTM as a highly-efficient candidate for ultra-low-power speech applications.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of TsetlinKWS, a novel algorithm-hardware co-design framework that significantly enhances the performance and efficiency of Tsetlin Machines for keyword spotting tasks. This work represents a meaningful advancement in the field of low-power machine learning accelerators, combining innovative methodologies with practical experimental validation.
Methodology
The paper introduces a novel algorithm-hardware co-design framework for the Convolutional Tsetlin Machine (CTM) specifically tailored for keyword spotting tasks. The methodology is robust, combining a new Mel-Frequency Spectral Coefficient and Spectral Flux (MFSC-SF) feature extraction scheme with an optimized grouped block-compressed sparse row (OG-BCSR) algorithm for model size reduction. The state-driven architecture proposed is well thought out, leveraging data reuse and sparsity effectively to enhance energy efficiency. The integration of these components demonstrates a comprehensive approach to improving the performance of Tsetlin Machines in speech applications.
Results & evaluation
The experimental results are compelling, showcasing a competitive accuracy of 87.35% on the 12-keyword spotting task, which is a significant achievement for Tsetlin Machines in this domain. The evaluation is performed using a 65nm process technology, with detailed measurements of power consumption (16.58 µW) and core area (0.63 mm²). The paper includes comparisons with state-of-the-art keyword spotting accelerators, highlighting the advantages of the proposed system in terms of logic operations per inference (907k) and overall efficiency.
Reproducibility
While the paper provides a thorough description of the methodologies and experimental setups, it lacks specific implementation details that would facilitate reproducibility. The absence of a project URL or demo further limits the ability of other researchers to replicate the results or build upon this work.
Limitations
One limitation is the focus on a specific task (12-keyword spotting), which may not generalize well to other speech recognition tasks. Additionally, while the energy efficiency is impressive, the trade-offs in terms of accuracy and model complexity compared to other methods could be further explored. The paper does not address potential scalability issues when applied to larger datasets or more complex tasks.
Broader impact
The proposed TsetlinKWS framework has significant implications for ultra-low-power speech applications, particularly in edge computing scenarios where energy efficiency is paramount. The advancements in model compression and hardware acceleration could pave the way for more widespread adoption of Tsetlin Machines in real-time applications, potentially impacting areas such as smart devices, IoT, and assistive technologies. The main contribution of this paper is the introduction of TsetlinKWS, a novel algorithm-hardware co-design framework that significantly enhances the performance and efficiency of Tsetlin Machines for keyword spotting tasks. This work represents a meaningful advancement in the field of low-power machine learning accelerators, combining innovative methodologies with practical experimental validation.
Author of this paper? Send a correction.