DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition
Abstract
Acoustic side-channel attacks (ASCA) on keyboards pose a significant security risk, as keystrokes can be inferred from typing acoustics, revealing sensitive information. Prior ASCA studies are limited by small-scale datasets with restricted diversity in users, keyboards, and environments, constraining analysis across devices, microphones, and noise conditions. We introduce HEAR, a dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias. HEAR contains recordings from 53 participants using 37 laptop keyboards, collected in three realistic settings: (1) external microphone capture, (2) device microphone capture without network noise, and (3) VoIP-based streaming capture. This enables controlled evaluation across users, keyboards, and environments. On HEAR, we establish an ASCA benchmark spanning conventional features and pre-trained representations from raw audio and spectrograms in unimodal and multimodal settings. We propose DECKER, a domain-invariant keystroke inference framework with four stages: (1) Keyboard Signature Normalization to reduce device coloration, (2) domain-adversarial disentanglement to suppress keyboard identity, (3) supervised cross-keyboard contrastive alignment to enforce key consistency, and (4) Acoustic Style Randomization to synthesize unseen keyboard responses. We further explore sentence-level inference using an LLM-based post-processing layer to refine keystroke sequences via linguistic context. Results on HEAR show DECKER improves keystroke identification over strong baselines, particularly in cross-keyboard and cross-user settings, with further gains from language-model rectification. These findings highlight that ASCA remains effective across diverse users, devices, and noisy environments, underscoring its practical security risk.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces DECKER, a novel domain-invariant framework for keystroke inference from acoustic side channels, demonstrating significant advancements in the generalization of ASCA techniques across diverse environments and user demographics. The comprehensive methodology and experimental validation position this work as a substantial contribution to the field of machine learning and security.
Methodology
The methodology presented in this paper is robust and innovative, particularly in the design of the DECKER framework, which integrates multiple advanced techniques such as Keyboard Signature Normalization, domain-adversarial disentanglement, and LLM-assisted decoding. The four-stage approach to keystroke inference is well-structured, allowing for domain-invariant representations that significantly enhance generalization across different keyboards and environments. The use of a comprehensive dataset (HEAR) that captures diverse user demographics and environmental conditions further strengthens the methodology by enabling realistic evaluations of ASCA.
Results & evaluation
The experimental evaluation is thorough, utilizing the HEAR dataset to establish a new benchmark for ASCA. Results demonstrate significant improvements in keystroke identification accuracy across various settings, particularly in cross-keyboard and cross-user scenarios. The paper provides detailed performance metrics, including key-level accuracy and sentence-level reconstruction, showcasing the effectiveness of the proposed methods against strong baselines. The inclusion of ablation studies and robustness analyses adds depth to the evaluation, confirming the contributions of each component of the DECKER framework.
Reproducibility
The paper outlines a clear and detailed implementation of the proposed methods, including training schedules, hyperparameters, and data collection processes. The authors provide a project URL for access to the dataset and experimental setup, which enhances reproducibility. However, the actual code repository is not mentioned, which could limit the ease of replication for other researchers.
Limitations
While the paper presents a comprehensive approach, it acknowledges limitations such as the potential for overfitting to specific user characteristics and the need for further exploration of countermeasures against ASCA. Additionally, the reliance on LLMs for sequence reconstruction may introduce computational overhead, which could be a concern for real-time applications.
Broader impact
The findings of this research have significant implications for security and privacy, particularly in understanding the vulnerabilities associated with acoustic side-channel attacks. The development of the HEAR dataset and the DECKER framework could pave the way for further research in ASCA and related fields, potentially influencing the design of more secure input methods and countermeasures against such attacks. The integration of LLMs into the ASCA framework also highlights the intersection of machine learning and security, opening avenues for future explorations in adversarial machine learning. The paper introduces DECKER, a novel domain-invariant framework for keystroke inference from acoustic side channels, demonstrating significant advancements in the generalization of ASCA techniques across diverse environments and user demographics. The comprehensive methodology and experimental validation position this work as a substantial contribution to the field of machine learning and security.
Author of this paper? Send a correction.