Hardware-accelerated graph neural networks: an alternative approach for neuromorphic event-based audio classification and keyword spotting on SoC FPGA
Abstract
As the volume of data recorded by embedded edge sensors increases, particularly from neuromorphic devices producing discrete event streams, there is a growing need for hardware-aware neural architectures that enable efficient, low-latency, and energy-conscious local processing. We present an FPGA implementation of event-graph neural networks for audio processing. We utilise an artificial cochlea that converts time-series signals into sparse event data, reducing memory and computation costs. Our architecture was implemented on a SoC FPGA and evaluated on two open-source datasets. For classification task, our baseline floating-point model achieves 92.7% accuracy on SHD dataset - only 2.4% below the state of the art - while requiring over 10x and 67x fewer parameters. On SSC, our models achieve 66.9-71.0% accuracy. Compared to FPGA-based spiking neural networks, our quantised model reaches 92.3% accuracy, outperforming them by up to 19.3% while reducing resource usage and latency. For SSC, we report the first hardware-accelerated evaluation. We further demonstrate the first end-to-end FPGA implementation of event-audio keyword spotting, combining graph convolutional layers with recurrent sequence modelling. The system achieves up to 95% word-end detection accuracy, with only 10.53 microsecond latency and 1.18 W power consumption, establishing a strong benchmark for energy-efficient event-driven KWS.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The main contribution of this paper is the introduction of a hardware-accelerated event-graph neural network for audio classification and keyword spotting, which demonstrates significant improvements in performance and efficiency compared to existing methods. The comprehensive analysis of the technical contribution, methodology, and significance to the field highlights the potential for advancing audio processing in resource-constrained environments.
Methodology
The paper presents a novel approach to audio classification and keyword spotting using event-graph neural networks implemented on an FPGA. The methodology is well-structured, leveraging an artificial cochlea to convert time-series signals into sparse event data, which is then processed using graph convolutional layers. The authors introduce several optimizations for hardware implementation, including a novel graph generation method that reduces memory usage and latency. The use of a skip step connection for neighbor searches is particularly innovative, as it addresses computational complexity effectively. However, the paper could benefit from a more detailed discussion on the limitations of the proposed methods and potential trade-offs involved in the hardware implementation.
Results & evaluation
The experimental evaluation is robust, with the authors conducting extensive tests on two open-source datasets (SHD and SSC). They provide a comprehensive analysis of their model's performance against state-of-the-art approaches, demonstrating significant improvements in accuracy, resource utilization, and latency. The results are well-presented, with clear metrics for both classification and keyword spotting tasks. However, the paper lacks a thorough comparison with other non-FPGA-based implementations, which would provide a clearer picture of the advantages of their approach.
Reproducibility
The paper outlines the implementation details and experimental setup, including the hardware used (Zynq UltraScale+ ZCU104 FPGA platform) and the software framework (PyTorch with PyTorch Lightning). However, the absence of a publicly available code repository or demo URL limits reproducibility. Future work should include making the code available to facilitate validation and further research.
Limitations
One limitation is the reliance on specific datasets, which may not generalize well to other audio processing tasks. Additionally, the complexity of the hardware implementation may pose challenges for practical deployment in diverse environments. The paper does not address potential scalability issues when adapting the architecture for larger datasets or more complex audio tasks.
Broader impact
The proposed method has significant implications for the development of energy-efficient audio processing systems, particularly in the context of IoT and edge devices. The ability to perform real-time audio classification and keyword spotting with low latency and power consumption could enhance applications in smart home devices, wearable technology, and robotics. As the demand for efficient processing in embedded systems grows, this research could pave the way for more advanced audio processing solutions. The main contribution of this paper is the introduction of a hardware-accelerated event-graph neural network for audio classification and keyword spotting, which demonstrates significant improvements in performance and efficiency compared to existing methods. The comprehensive analysis of the technical contribution, methodology, and significance to the field highlights the potential for advancing audio processing in resource-constrained environments.
Author of this paper? Send a correction.