谷歌用鸟类声学模型识别鲸鱼叫声

原标题:How AI trained on birds is surfacing underwater mysteries
42AI 研判分

谷歌研究人员利用在鸟类声音上训练的深度学习模型,成功识别出多种鲸鱼的复杂叫声。该方法通过迁移学习解决了海洋生物声学数据标注稀缺的问题,并协助 NOAA 确认了神秘“biotwang”声源为布氏鲸。

报道正文

本文目录5 个章节

水下声学对于理解海洋物种及其环境中不可见的规律至关重要。海洋声景充满了神秘的声音和尚未发现的秘密。例如,神秘的“biotwang”声音最近被美国国家海洋和大气管理局(NOAA)归因于难以捉摸的布氏鲸,这一发现凸显了持续识别新歌声类型及物种归属的挑战。

谷歌长期以来一直与外部科学家合作,利用生物声学监测和保护鲸类,包括我们最初用于检测座头鲸分类的研究模型,以及 2024 年发布的多物种鲸类模型(详见相关发布文章)。为了保持这一发展步伐,谷歌在生物声学 AI 领域的方法正在不断演进,旨在更高效地将新发现转化为大规模的科学洞察。2025 年 8 月,Google DeepMind发布了最新的Perch 基础生物声学模型——Perch 2.0。这是一个主要基于鸟类及其他陆生发声动物训练的生物声学基础模型。令人惊讶的是,尽管训练数据中未包含任何水下音频,Perch 2.0 作为嵌入模型在海洋验证任务的迁移学习中表现优异。

在我们最新的论文“Perch 2.0 将‘鲸鱼’任务迁移至水下声学”中,这项由 Google Research 与 Google DeepMind 合作完成的研究在 NeurIPS 2025 的 AI for Non-Human Animal Communications(面向非人类动物交流的 AI)研讨会上进行了展示。我们在文中深入探讨了这些成果,展示了这个主要基于鸟类数据训练的生物声学基础模型如何被用于赋能并扩展对水下海洋生态系统的洞察,特别是在鲸类发声分类方面。此外,我们还分享了一个端到端的 Google Colab 教程,演示我们的敏捷建模工作流:即如何利用 Perch 2.0,通过 Google Cloud 访问 NOAA NCEI 被动声学数据档案库,为鲸类发声创建自定义分类器。

生物声学分类的工作原理

如果现有的预训练分类模型(例如我们的多物种鲸类模型)已经具备所需的标签类别,并且在研究人员的数据集上表现良好,那么可以直接使用该模型为其音频数据生成评分和标签。然而,若要为全新发现的声音创建新的自定义分类器,或提升在新数据上的准确率,我们可以利用 迁移学习 而非从零开始构建新模型。这种方法大幅减少了创建新自定义分类器所需的计算量和实验次数。

在生物声学的迁移学习中,预训练模型(如 Perch 2.0)用于为每个音频窗口生成嵌入向量(embeddings)。这些嵌入向量将庞大的音频数据压缩为一组更小的特征数组,作为简单分类器的输入。要为任意一组带标签的音频数据创建新的自定义模型,我们首先应用预训练模型处理音频数据以获取嵌入向量,然后将这些嵌入向量作为逻辑回归分类器的输入特征。相比学习深度神经网络的所有参数,我们现在只需学习逻辑回归最后一步的新参数,这在研究人员的时间和计算资源上都更加高效。

评估

我们使用少样本线性探针(few-shot linear probe)在海洋任务上评估了 Perch 2.0,例如区分不同的须鲸物种或不同的虎鲸亚种群。其性能与我们的 Perch Hoplite 仓库中支持的预训练模型进行了对比,该仓库旨在支持敏捷建模和迁移学习。这些模型包括 Perch 2.0、Perch 1.0、SurfPerch 以及多物种鲸鱼模型。

在水下数据评估方面,我们使用了三个数据集:NOAA PIPAN、ReefSet 和 DCLDE。

  • NOAA PIPAN:来自 NOAA 太平洋群岛渔业科学中心录音的 NOAA NCEI 被动声学数据档案 的带标注子集。该数据集包含我们先前鲸类模型中使用的标签,以及针对须鲸物种(如普通小须鲸、座头鲸、塞鲸、蓝鲸、长须鲸和布氏鲸)的新增标注。
  • ReefSet:为 SurfPerch 模型的训练而开发,该数据集利用了 Google 艺术与文化项目——“召唤我们的珊瑚”——中的数据标注。它涵盖了混合的生物礁石噪声(如咯咯声、噼啪声、咆哮声)、特定物种/属类别(例如雀鲷、海豚和石斑鱼),以及人为噪声和海浪类别。
  • DCLDE:该数据集使用三组不同的标签集进行评估:
    • Species(物种):用于区分虎鲸、座头鲸、非生物声音以及未知水下声音(其中虎鲸和座头鲸的标签存在一定的不确定性)。
    • Species Known Bio(已知生物物种):用于确定虎鲸和座头鲸的标签。
    • Ecotype(生态型):用于区分虎鲸亚种群(生态型),包括 transient/Biggs 型、北部居民型、南部居民型、东南阿拉斯加虎鲸以及远洋型虎鲸。

在本协议中,对于给定的带有标注数据的目标数据集,我们从每个候选模型中提取嵌入向量(embeddings)。随后,我们为每个类别选取固定数量的样本(4、8、16 或 32 个),并在这些嵌入向量之上训练一个简单的多类逻辑回归模型。我们利用生成的分类器计算 受试者工作特征曲线下面积(AUC_ROC),数值越接近 1,表明区分不同类别的能力越强。这一过程模拟了如何利用预训练的嵌入模型,仅凭少量标注样本构建自定义分类器。

[注:本文篇幅超长,以上为核心前篇精译,后续内容保留原文呈现]

Our results show that more examples per class improve performance across all the models, except on ReefSet data, where performance is high even with only four examples per class for all models, except the multispecies whale model. Notably, Perch 2.0 is consistently either the top or second-best performing model for each dataset and sample size.

We also compare Perch 2.0 to embeddings from AVES-bird and AVES-bio (Earth Species Project transformer bioacoustics models trained on birds and biological sounds, respectively) and BirdNet v2.3 from K. Lisa Yang Center for Conservation Bioacoustics at theCornell Lab of Ornithology. Perch 2.0 outperforms AVES-bird and AVES-bio on most underwater tasks, but there are other pre-trained models that also perform well that weren’t trained on underwater audio.

How does Perch 2.0 work so 'whale'?

We offer a few possible reasons for this transfer performance from our model trained primarily on birds to underwater sounds. First, prior research shows that larger models with extensive training data generalize better, allowing our bioacoustics model to perform well even on downstream tasks classifying sounds for species and sounds not included in the training dataset. Additionally, the challenge of classifying similar bird calls (the “bittern lesson”) forces the model to learn detailed acoustic features that can then be informative for other bioacoustics tasks. For example, there are 14 species of doves in North America, each with their own subtly distinct “coo” sound. A model that extracts the features that can distinguish between each species-specific “coo” is likely to isolate features that can help separate other sound classes. Finally, feature transfer across different types of species could also be related to the sound production mechanism itself, where a variety of species — including birds and marine mammals — have evolved similar means of sound production.

A high performing model will have embeddings that are informative and linearly separable for the applied target classes. To visualize, we plot a summary of embeddings from each model using a procedure called tSNE, where different colors represent different classes. A highly informative model will show distinct clusters for each class, whereas the classes will be more intermixed in a less informative model (such as the Google multi-species whale model). While almost all models show some distinct clusters of points for Southern Resident killer whales (KW_SRKW) and Southern Alaskan Residents (KW_SAR), the resulting embeddings of sounds from Northern Resident killer whales (KW_NRKW), Transient killer whales (KW_TKW), and Offshore killer whales (KW_OKW) are intermingled in models such as AVES-bio, AVES-bird, and SurfPerch, but are more clearly distinguished in BirdNet v2.3 and Perch 2.0.

Looking ahead

The Google DeepMind Perch team, in collaboration with Google Research and external partners, has pioneered an agile modeling approach for bioacoustics to create a custom classifier from a small number of labelled examples within a couple hours. In order to support both Google Research partners as well as the broader cetacean acoustics community, we have created an end-to-end demo for working with the NOAA data from the Passive Acoustic Archive dataset hosted on Google Cloud, updating our prior tutorials using the more efficient Perch Hoplite databases for managing embeddings.

Acknowledgements

The Perch Team, which developed the Perch 2.0 model and is part of Google DeepMind, includes Tom Denton, Bart van Merriënboer, Vincent Dumoulin, Jenny Hamer, Isabelle Simpson, Andrea Burns, and Lauren Harrell (Google Research). Special thanks to Ann Allen (NOAA Pacific Islands Fisheries Center) and Megan Wood (Saltwater Inc. in support of NOAA) for providing additional annotations used in the NOAA PIPAN dataset, Dan Morris (Google Research) and Matt Harvey (Google DeepMind).