Generating the Unheard: Phylogeny-Guided Latent Generation for Ancestral Sound Reconstruction
Abstract
What did an ancestral bird species sound like? Existing ancestral state reconstruction methods can infer low-dimensional traits such as morphological characters at internal nodes of a phylogenetic tree, but no one has tried to produce rich perceptual signals such as audio. Some of the challenges include inferred representations that are either too low-dimensional to decode or lie in non-generative feature spaces, so no method to date can produce ancestral audio. We introduce the first framework that generates plausible ancestral vocalizations. Our pipeline encodes bird recordings into a VAE latent space, learns a low-dimensional trait projection aligned with phylogenetic distances, performs ancestral inference in this trait space, and recovers decodable latents through an anchored inverse lift before emitting novel waveforms for each ancestral node. Because the entire pipeline stays within a decodable latent space, every internal node receives a genuinely new audio output representing plausible intermediate ancestral sounds unavailable to retrieval-based alternatives. Experiments on two phylogenetically distant bird clades, 21-species Tyrannidae and 19-species Paridae, show that our method is the only approach that simultaneously achieves genuine generation, phylogenetic consistency, and naturalistic audio quality across both datasets.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces the first framework for generating plausible ancestral vocalizations by combining phylogenetic inference with generative audio models. It effectively addresses the challenge of reconstructing high-dimensional perceptual signals on phylogenies by using a learned trait subspace and an anchored inverse lift to ensure decodable, naturalistic audio outputs that are phylogenetically consistent.
Methodology
The paper proposes a novel four-stage pipeline for ancestral sound reconstruction: encoding audio into a VAE latent space, learning a low-dimensional tree-metric trait projection, performing Brownian motion ancestral inference in this trait space, and recovering decodable latents via an anchored inverse lift. The key technical contribution is the "anchored inverse lift," which resolves the underdetermined inverse problem of mapping from a low-dimensional trait space back to the high-dimensional VAE latent space by preserving the nullspace component (acoustic texture) from a nearby observed latent. This is mathematically grounded with proofs for exact trait preservation and nullspace preservation. The use of Cholesky reparametrization for the Stiefel manifold constraint is a solid optimization choice.
Results & evaluation
The experiments are conducted on two distinct bird clades (Tyrannidae and Paridae) to test generalization across suboscine (innate) and oscine (learned) vocalizations. The evaluation metrics are well-chosen for a task without ground truth, focusing on phylogenetic plausibility (Tree-Embedding Correlation, Edge Monotonicity) and audio quality (FAD, Silence Rate). The results clearly demonstrate that the proposed method outperforms naive latent-space BM in audio quality and retrieval baselines in novelty (Unique Rate). The ablation study effectively isolates the contribution of the learned projection and the anchoring strategy.
Reproducibility
The paper provides detailed algorithmic descriptions, hyperparameters, and dataset curation steps. It specifies the use of pretrained Make-An-Audio 2 VAE and BigVGAN vocoder, which are publicly available. However, the specific code for the trait learning and ancestral inference pipeline is not linked in the provided text, which slightly hinders immediate reproducibility, though the mathematical formulation is clear enough for re-implementation.
Limitations
The primary limitation is the lack of ground truth for ancestral sounds, making evaluation inherently indirect. The method relies on the assumption that Brownian motion is a suitable model for vocal evolution, which may not hold for all traits or clades, although the paper does test alternative models (OU, Pagel's lambda) in the appendix. The anchor dependence means that fine-grained acoustic texture is copied from extant species rather than inferred, which could introduce bias if the nearest anchor is not representative.
Broader impact
This work bridges evolutionary biology and machine learning, offering a tool to visualize and analyze ancestral traits in a perceptual modality. It could impact comparative biology, conservation efforts (reconstructing extinct species' calls), and the understanding of vocal evolution and speciation. The framework is generalizable to other high-dimensional traits where phylogenetic signal exists but direct generation is challenging. The paper introduces the first framework for generating plausible ancestral vocalizations by combining phylogenetic inference with generative audio models. It effectively addresses the challenge of reconstructing high-dimensional perceptual signals on phylogenies by using a learned trait subspace and an anchored inverse lift to ensure decodable, naturalistic audio outputs that are phylogenetically consistent.
Author of this paper? Send a correction.