PACE: A Playback-Aligned Context Engine for LLM-Based Full-Duplex Voice Dialogue
Abstract
LLM-based full-duplex voice services allow users to speak while the assistant is responding. Because servers can generate output and advance dialogue state faster than clients can play it, subsequent user speech may be interpreted based on content the user never heard. We call this failure Generative Context Mis-anchoring (GCM). To address GCM issues, we present PACE, a provider-independent middleware layer that anchors model-facing context to the client playback boundary, a system-observable proxy for what the user could have heard. After an interruption, PACE repairs this context to exclude assistant content that never reached playback, while preserving low-latency generation across heterogeneous voice runtimes. We implement PACE's audio-only projection path end to end in a browser-based realtime voice assistant using a black-box speech model, without modifying the model service. We also construct GCM-Bench, a new controlled benchmark dataset of 108 playback-relative referent-anchoring cases. On GCM-Bench, PACE raises Referent Anchoring Accuracy from 25.0% to 96.3% over a cancellation-only baseline. On 200 Full-Duplex-Bench v1 interruption samples, it preserves interruption response quality. These results show that grounding model-facing context in actual playback is a practical way to maintain consistency in full-duplex voice dialogue.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a practical systems solution for context misalignment in full-duplex voice LLMs, demonstrating significant accuracy improvements on a new benchmark through playback-aware context repair.
Methodology
The paper introduces PACE, a middleware layer designed to solve "Generative Context Mis-anchoring" (GCM) in full-duplex voice LLM systems. The core innovation is decoupling the model's internal dialogue state from the client's actual playback boundary. By tracking audio samples rendered to the user, PACE identifies which assistant content was actually heard. Upon interruption, it "repairs" the context by revoking unplayed content before passing the user's input to the model. This is a systems-level architectural contribution rather than a novel algorithmic or modeling breakthrough. The approach is pragmatic, leveraging existing APIs (like OpenAI's item truncation) but providing a generalized abstraction for state management across heterogeneous runtimes. The methodology is sound for its specific problem domain but relies on standard software engineering patterns (state tracking, event handling) rather than new machine learning techniques.
Results & evaluation
The authors evaluate PACE using two benchmarks. First, GCM-Bench, a new dataset of 108 cases specifically designed to test referent anchoring accuracy. PACE improves accuracy from 25.0% to 96.3% compared to a baseline that only cancels generation without context repair. This is a significant and clear improvement, demonstrating the necessity of context repair. Second, they evaluate on 200 samples from Full-Duplex-Bench v1, showing that PACE preserves interruption response quality (mean score 4.975 vs 4.995). The evaluation is relevant and directly addresses the proposed problem. However, the GCM-Bench is small (108 cases) and likely synthetic or narrowly curated, which limits the generalizability of the accuracy claims. The latency impact is mentioned but not deeply quantified in terms of added overhead, which is critical for real-time voice systems.
Reproducibility
The paper provides a link to a GitHub repository for the GCM-Bench dataset. The implementation is described as a browser-based middleware. While the code for the benchmark is available, the full source code for the PACE middleware itself is not explicitly linked in the provided text (only the benchmark repo is cited). The description of the "audio-only projection path" is detailed enough for a systems researcher to replicate the logic, but the exact integration with specific black-box models might require additional inference. The use of a standardized benchmark (FDB v1) aids in reproducibility of the interruption quality metrics.
Limitations
The primary limitation is the scope of the evaluation. GCM-Bench is small and potentially lacks the diversity of natural conversation nuances found in larger, open-domain datasets. The paper focuses on a "black-box" projection path; while this is a strength for portability, it may not achieve the same precision as a system with deep access to internal model states (like KV-cache truncation), which the paper acknowledges but does not benchmark against in terms of ultimate accuracy ceiling. Additionally, the paper does not extensively discuss the computational overhead of the middleware itself, which could be non-trivial in high-concurrency scenarios. The assumption that playback boundary is a perfect proxy for "what the user heard" ignores user attention and cognitive processing, though the authors correctly note this distinction.
Broader impact
This work addresses a critical usability issue in the deployment of full-duplex voice assistants. As LLM-based voice interfaces become more common, ensuring that the system's context aligns with the user's actual experience is vital for trust and effectiveness. PACE provides a practical, provider-independent solution that can be adopted by various voice service providers. It highlights the importance of systems-level design in multimodal AI, showing that even with powerful models, poor state management can lead to significant failures. This could influence the design of future voice assistant frameworks to prioritize playback-aware context management. The paper presents a practical systems solution for context misalignment in full-duplex voice LLMs, demonstrating significant accuracy improvements on a new benchmark through playback-aware context repair.
Author of this paper? Send a correction.