Positive-Unlabelled Active Learning to Curate a Dataset for Orca Resident Interpretation
Abstract
This work presents the largest curation of Southern Resident Killer Whale (SRKW) acoustic data to date, also containing other marine mammals in their environment. We systematically search all available public archival hydrophone data within the SRKW habitat (over 30 years of audio data). The search consists of a weakly-supervised, positive-unlabelled, active learning strategy to identify all instances of marine mammals. The resulting transformer-based detectors outperform state-of-the-art detectors on the DEEPAL, DCLDE-2026, and two newly introduced expert-annotated datasets in terms of accuracy, energy efficiency, and speed. The detection model has a specificity of 0-28.8% at 95% sensitivity. Our multiclass species classifier obtains a top-1 accuracy of 42.1% (11 train classes, 4 test classes) and our ecotype classifier obtains a top-1 accuracy of 43.0% (4 train classes, 5 test classes) on the DCLDE-2026 dataset. We yield 919 hours of SRKW data, 230 hours of Bigg's orca data, 1374 hours of orca data from unlabelled ecotypes, 1501 hours of humpback data, 88 hours of sea lion data, 246 hours of pacific white-sided dolphin data, and over 784 hours of unspecified marine mammal data. This SRKW dataset is larger than DCLDE-2026, Ocean Networks Canada, and OrcaSound combined. The curated species labels are available under CC-BY 4.0 license, and the corresponding audio data are available under the licenses of the original owners. The comprehensive nature of this dataset makes it suitable for unsupervised machine translation, habitat usage surveys, and conservation endeavours for this critically endangered ecotype.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a significant advancement in the curation of marine mammal acoustic data, leveraging innovative machine learning techniques to enhance conservation efforts for critically endangered species. The combination of active learning and citizen science not only improves model performance but also broadens the scope of participation in scientific research, making it a noteworthy contribution to both machine learning and marine biology.
Methodology
The paper introduces a novel weakly-supervised, positive-unlabelled active learning strategy to curate a large dataset of Southern Resident Killer Whale (SRKW) acoustic data. The methodology effectively combines active learning with citizen science, allowing for the identification and classification of marine mammal sounds from a vast archive of unlabelled audio data. The use of transformer-based detectors is a significant advancement over previous models, and the paper provides a detailed explanation of the active learning strategies employed, including entropy-based and mixed sampling approaches.
Results & evaluation
The experiments are comprehensive, comparing the proposed models against state-of-the-art detectors on multiple datasets. The results demonstrate that the new models not only outperform existing techniques in terms of accuracy, energy efficiency, and speed but also provide a detailed analysis of the classifiers' performance across various datasets. The inclusion of a citizen science component adds an interesting dimension to the evaluation, showcasing the potential for non-expert contributions to improve model performance.
Reproducibility
The paper provides sufficient details regarding the methodology, including data preprocessing, model training, and evaluation metrics, which supports reproducibility. However, the lack of a publicly available code repository limits the ease of reproducing the results. The authors do mention the use of Hugging Face for dataset access, which is a positive aspect for reproducibility.
Limitations
The study acknowledges potential limitations, such as the reliance on citizen science for data labelling, which may introduce noise. Additionally, the dataset's sampling bias due to hydrophone placements is noted, which could affect the generalizability of the findings. The paper also mentions that the models are designed specifically for certain frequencies, which may limit their applicability to other marine species.
Broader impact
The curated dataset and the methodologies developed have significant implications for marine biology, conservation efforts, and bioacoustics research. The availability of a large, openly licensed dataset can facilitate further research into marine mammal behavior and ecology, while the active learning approach can inspire similar methodologies in other fields requiring large-scale data analysis. The work also highlights the potential for citizen science to contribute meaningfully to scientific research. This paper presents a significant advancement in the curation of marine mammal acoustic data, leveraging innovative machine learning techniques to enhance conservation efforts for critically endangered species. The combination of active learning and citizen science not only improves model performance but also broadens the scope of participation in scientific research, making it a noteworthy contribution to both machine learning and marine biology.
Author of this paper? Send a correction.