Hugging Face 发布 WebGPU 内核库

原标题:Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI
68AI 研判分

Hugging Face 推出 @huggingface/kernels 库及包含 207 个优化内核的集合,旨在加速浏览器端本地 AI 推理。该工具通过提供底层 GPU 操作支持,简化了 WebGPU 在模型执行中的集成与性能调优。开发者可直接复用这些内核以提升网页应用的推理速度与兼容性。

关联知识实体

报道正文

本文目录8 个章节

返回文章列表

推出 @huggingface/kernels:面向本地 AI 的 200+ WebGPU 内核

发布于 2026 年 9 月 1 日 在 GitHub 上更新 点赞 93

Nico Martin nico-martin Joshua Xenova

Hugging Face WebAI 团队的首要目标之一,是让浏览器端推理尽可能快速且易于使用。实现这一目标是一项多层面的工程:模型需要采用适合浏览器的表示形式,运行时需构建高效的执行计划,而位于技术栈底层的各个 GPU 操作则必须充分利用不同设备以及各类浏览器实现的特性。

今天,我们发布了这项工作的第一层成果:@huggingface/kernels,这是一个极简库,用于从 Hugging Face Hub 加载并运行经过优化的 WebGPU 内核;同时,我们在 huggingface.co/webgpu-kernels 上推出了包含 207 个内核的初始集合。

该集合涵盖了适用于多种机器学习架构和工作负载的操作。更重要的是,每个内核都作为一个完整的、带版本管理的包发布:其接口定义、着色器模板、正确性测试用例、基准测试用例以及使用说明均统一托管在 Hub 上。

我们还同步上线了 Fleet,这是一套基于浏览器的 GPU 基准测试与验证套件,可在你的硬件上运行内核并进行评分。除了提供你本机设备的测试结果外,Fleet 还为社区提供了一种途径,让我们能够收集来自传统测试实验室无法覆盖的设备上的性能与正确性证据。经用户授权后,每次运行都会生成私密数据,帮助我们发现故障(如结果错误、异常缓慢的情况等)、改进内核变体,并在真实世界硬件上做出更优的优化决策。

TL;DR

  • 207 个 WebGPU kernels,以独立仓库的形式发布在 webgpu-kernels 组织下。采用 Apache-2.0 许可证。
  • 一个 JavaScript 加载器 @huggingface/kernels,可直接从 Hub 下载、准备并运行 kernels。
  • 每个 kernel 都有明确的契约和可复现的证据,包括清单文件(manifests)、正确性测试、基准测试用例以及 WGSL shader 模板。
  • Fleet,一款基于浏览器的基准测试工具,通过众包方式收集真实世界 GPU 上的正确性和性能证据,帮助我们改进 kernels 及其变体。

为什么从 kernels 入手?

在浏览器中运行的模型最终会转化为一系列 GPU 操作:矩阵乘法、归一化、卷积、attention 原语、量化操作、数据布局转换等等。WebGPU 通过可移植 API 使这些操作在现代浏览器中可用,而 WGSL 则为执行这些操作的 shaders 提供了一种通用语言。

然而,可移植性并不自动等同于高性能。两个 shaders 可以实现相同的操作并产生相同的输出,但在不同的加速器上表现却截然不同。工作组大小(workgroup sizes)、内存访问模式、向量化、数据类型以及融合策略都会影响性能。最佳选择还会随着输入形状、设备、浏览器以及可用的 WebGPU 特性而变化。

这就是为什么 kernels 构成了快速浏览器推理的基础层。更高层级的运行时效率不可能超过其调度的底层操作。通过使这些操作具备单独的可发现性、可测试性、可基准测试性以及版本管理,我们可以独立地改进基础层,同时为上层保持稳定的契约。

是 kernel 仓库,而不仅仅是 shader

集合中的每个 kernel 都有其独立的仓库和 kernel card(内核卡片)。该卡片记录了操作的语义、输入、输出、属性、支持的数据类型、源文件,以及一个可直接运行的 @huggingface/kernels 示例。

例如,ai.onnx.Add 实现了带有双向广播的逐元素加法。它是神经网络中最简单的操作之一,广泛应用于从残差连接到偏置添加等各种场景。其卡片记录了两个输入、广播后的输出形状、支持的数据类型,以及针对不同形状和设备提供的变体。

ai.onnx.Add WebGPU 内核仓库中的文件

ai.onnx.Add 仓库将清单(manifest)、正确性与基准测试用例以及 WGSL 着色器模板打包在一起。

在卡片背后,该仓库包含了理解和评估实现所需的各类工件:

  • manifest.json 是操作契约的唯一事实来源。它定义了输入、输出、属性、类型约束和形状推导规则。
  • metadata.json 记录内核标识符、摘要(digests)和溯源信息。
  • test.json 包含正确性测试用例,以便根据预期行为检查实现。
  • bench.json 包含用于评估内核的基准测试和调优用例,代表了实际的工作负载。
  • *.wgsl.jinja 文件包含参数化的 WGSL 实现,用于为特定请求和设备生成着色器。

这种结构将着色器转化为可复用的软件工件。无需阅读 WGSL 代码即可检视接口,正确性和性能用例随实现一同分发,且发布的版本可以被显式加载,而不必依赖未版本化的文件 URL。我们的内核还可以作为参考实现,供构建自定义 WebGPU 内核或将这些操作集成到其自有运行时的开发人员使用。

从 Hub 加载内核

通过 npm 安装该包:

BASH
npm install @huggingface/kernels@preview

运行这些内核需要支持 WebGPU 的浏览器。WebGPU 的可用性取决于浏览器、操作系统、GPU 和驱动程序。你可以在 JavaScript 中通过 "gpu" in navigator 来检查是否支持。

@huggingface/kernels 提供了内核仓库与你的应用程序之间的桥梁。调用 getKernel 并传入 Hub 仓库 ID 和契约版本,然后使用带类型的输入数据和张量形状调用返回的函数。以下是一个简单的 bias-add 示例:

JAVASCRIPT
import { getKernel } from "@huggingface/kernels";

const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });

const { c } = await add({
  a: {
    data: new Float32Array([1, 2, 3, 4, 5, 6]),
    shape: [2, 3],
  },
  b: {
    data: new Float32Array([10, 20, 30]),
    shape: [3],
  },
});

The second input is broadcast across the first dimension, producing an output with shape [2, 3]. The loader derives that output shape and logical data type from the manifest contract and the inputs, then allocates c automatically.

Addition on six floats is deliberately the smallest possible demo. At this size, the GPU round trip costs far more than the math. The point is the call pattern: it stays exactly the same for the heavyweight operations where optimized kernels actually pay off, such as matrix multiplication (ai.onnx.MatMul). Only the repository ID and the inputs change.

Even this elementary operation illustrates why kernels need variants. Equal-shape addition can use a direct vectorized path, while broadcasted inputs need different indexing logic. The published Add kernel includes variants for equal shapes, vectorized broadcasting, scalar processing, and general broadcasting. The runtime can select an implementation that fits the current call and device without changing the application-facing API.

The version: 1 option selects version 1 of the published kernel contract. It is separate from an ONNX opset, an operator's since_version, or a model revision. Keeping those concepts separate lets applications depend on a stable JavaScript-facing contract while kernel implementations evolve behind it.

How fast are the kernels?

So, how much of a difference do optimized kernels actually make? We put our collection head-to-head with ORT WebGPU on an Apple M4 GPU, using ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a. We started with 1,756 test cases across all 207 operations and kept the 809 cases where both sides produced matching outputs and reliable timings.

Across those comparisons, our kernels were 2.57x faster by geometric mean and 1.90x faster at the median, with 629 wins, 176 losses, and 4 ties. Here is a closer look at four familiar operations:

OperationCompared casesOur WebGPU KernelORT WebGPUSpeedup
Add50.064 ms0.227 ms3.52x
MatMul290.115 ms0.131 ms1.14x
Softmax120.114 ms0.240 ms2.11x
LayerNormalization60.061 ms0.135 ms2.22x

[注:本文篇幅超长,以上为核心前篇精译,后续内容保留原文呈现]

Some individual wins were much bigger. A particularly difficult bilinear Einsum case (i,ij,j with size 4096) ran in 0.136 ms with our kernel versus 1,396 ms with ORT WebGPU: more than 10,000x faster. A row-wise CumSum over [256, 4096] was 301x faster, at 0.016 ms versus 4.784 ms. These are unusual cases rather than the speedups you should expect everywhere, but they show how much a specialized kernel can help when a general implementation hits a slow path.

We timed the work done on the GPU itself, leaving out setup such as loading kernels, creating sessions, uploading inputs, compiling shaders, and reading outputs back. Very short workloads are naturally harder to measure, and small cases can benefit from the GPU cache, so these numbers are best read as a useful comparison rather than a promise for every application.

They are also results for individual operations, not complete models. Exact performance will change across GPUs and browsers, which is why Fleet is so important for building a broader picture.

We are also working with the ONNX Runtime team to upstream these improvements so they can benefit the broader ONNX Runtime Web ecosystem.

From one device to a fleet

WebGPU performance varies across GPUs, browsers, and drivers, so results from one machine only tell part of the story. Fleet lets anyone run correctness and performance checks in the browser and see how the kernels behave on their hardware.

With consent, each run privately contributes evidence that helps us spot device-specific failures, compare variants, and improve selection rules. The goal is simple: use broad, real-world coverage to make the kernels faster and more reliable for everyone.

Building a shared foundation for WebAI

The initial 207 kernels are a starting point, not the end state. Publishing kernels independently on the Hub gives us a common place to inspect contracts, compare implementations, reproduce correctness checks, and improve performance without embedding every shader directly into every runtime.

The collection is also part of the Hub's broader kernel ecosystem: on the Kernels page, the WebGPU kernels sit alongside kernels for CUDA, ROCm, Metal, and other platforms, and can be filtered, sorted, and explored like any other artifact on the Hub.

The Hub Kernels page filtered to the WebGPU platform, listing the 207 published kernels

All 207 WebGPU kernels on the Hub's Kernels page, filtered by platform.

The pieces reinforce one another:

  1. Kernel repositories define transparent, versioned operation contracts.
  2. @huggingface/kernels makes those operations straightforward to load and run from JavaScript.
  3. Fleet crowdsources real-world evidence across a much broader range of devices than a conventional benchmark lab can cover.
  4. Every contributed run can reveal failures, guide tuning, improve variant selection, and help validate future kernel versions.

This is the low-level foundation for the next steps in our browser inference stack. We are excited to connect these kernels to higher-level model tooling, continue expanding operation coverage, and make fast local inference easier to use across the WebAI ecosystem.

Explore the WebGPU kernel collection, try @huggingface/kernels, and join the Fleet to contribute evidence from your device and help us make the kernels better for everyone.