PhysWave: Physics-Guided Latent Diffusion Models for Controllable Spatial Audio Generation
Abstract
Text-to-spatial audio generation, such as text-to-First-Order Ambisonics (FOA), provides a convenient way to create spatial audio for billion-dollar gaming and film industries. However, existing text-to-FOA methods are largely data-driven and may produce audio that violates acoustic relations between source direction and distance. They also separate descriptive and parametric control, forcing users to trade usability for precision. In this paper, we present PhysWave, a physics-guided latent diffusion model for controllable text-to-FOA generation. PhysWave unifies natural-language and trajectory control through a shared waypoint-caption representation, and augments diffusion training with two differentiable acoustic priors: spherical-harmonic direction consistency and inverse-square distance consistency. To support dynamic spatial generation, we further construct a 300K-clip FOA dataset with diverse sound categories and source trajectories. Extensive results show that the proposed priors help PhysWave generate spatially consistent FOA audio while maintaining competitive audio quality. Further analyses show that these physics priors improve spatial consistency during training and can also be used as inference-time guidance for training-free spatial refinement.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents PhysWave, a physics-guided latent diffusion model that improves spatial consistency in text-to-FOA generation by incorporating differentiable acoustic priors for direction and distance, along with a unified control interface for natural language and parametric trajectories.
Methodology
The paper proposes PhysWave, a physics-guided latent diffusion model for text-to-First-Order Ambisonics (FOA) generation. The core methodological contribution is the integration of differentiable acoustic priors (spherical-harmonic direction consistency and inverse-square distance consistency) into the diffusion training objective. By decoding the predicted clean latent back to the waveform during training and applying these physical constraints, the model explicitly enforces spatial accuracy rather than relying solely on implicit data-driven learning. Additionally, the paper introduces a unified waypoint-caption representation that allows both natural language descriptions and precise parametric trajectories to condition the same model, bridging the gap between usability and precision.
Results & evaluation
The authors construct a 300K-clip synthetic FOA dataset, which is a significant resource contribution. Experiments include ablations on the physics losses, comparisons with state-of-the-art methods (ImmerseDiffusion, SonicMotion), and subjective listening tests. The results show that the physics priors significantly reduce angular error and improve distance consistency without degrading audio quality metrics (FAD, CLAP). The subjective evaluation with 14 participants confirms the perceptual benefits.
Reproducibility
The paper provides detailed implementation details, including the VAE architecture, diffusion model hyperparameters, and the specific prompts used for the LLM parser. The release of the 300K dataset and the project website enhances reproducibility. However, the reliance on a specific LLM (Qwen3.5-4B) for parsing might introduce variability, though the deterministic post-processing helps.
Limitations
The model currently focuses on single-source trajectories in a free-field setting, ignoring room effects like reverberation and occlusion. It also does not handle multi-source scenes. The synthetic nature of the dataset, while large, may not fully capture the complexity of real-world spatial audio recordings.
Broader impact
This work has significant potential for the gaming, film, and VR industries by enabling non-experts to create high-quality spatial audio via text. The physics-guided approach could be extended to other domains where physical constraints are known, such as video generation or scientific simulation. The paper presents PhysWave, a physics-guided latent diffusion model that improves spatial consistency in text-to-FOA generation by incorporating differentiable acoustic priors for direction and distance, along with a unified control interface for natural language and parametric trajectories.
Author of this paper? Send a correction.