@huggingface/kernels reports 2.57x faster WebGPU operations. Measure the whole browser
Hugging Face's new @huggingface/kernels package loads optimized WebGPU kernels from the Hub into JavaScript applications. It is a preview release with 207 operations. Each operation comes with its interface, shader templates, correctness cases, benchmark cases, and instructions.
The launch comparison reports a 2.57x geometric-mean speedup over ONNX Runtime Web, with a 1.90x median. Both figures describe individual GPU operations on one Apple M4 GPU. Neither tells us how long a person waits for a model's answer.
Faster kernels can improve local AI. A browser still has to download artifacts, acquire a device, compile shaders, upload inputs, dispatch many operations, and read the outputs. Measure the kernel separately from the product around it.
A kernel is now a package, not a loose shader
A WebGPU model eventually becomes a chain of operations such as matrix multiplication, normalization, convolution, attention, quantization, and layout conversion. WGSL lets browser implementations consume a common shader language. It cannot make equivalent shaders equally fast. Workgroup size, memory access, vectorization, data types, input shapes, GPU, driver, and browser all affect the result.
Each operation lives in its own Hub repository. The contract is in manifest.json, covering inputs, outputs, attributes, type constraints, and shape rules. metadata.json holds identifiers, digests, and provenance. The repository also carries correctness cases in test.json, benchmark and tuning cases in bench.json, and parameterized WGSL in *.wgsl.jinja files.
The loader call is intentionally small:
import { getKernel } from "@huggingface/kernels";
const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });
In this call, version: 1 selects the published JavaScript-facing contract. It is distinct from an ONNX opset, an operator's since_version, and a model revision. Implementations can therefore change behind a stable application interface.
The announcement installs the library with npm install @huggingface/kernels@preview. Both the preview and latest npm tags currently resolve to 0.0.1-preview.1. Pin that exact package version in a serious experiment. Otherwise the preview tag can move while the application code stays unchanged.
The denominator behind 2.57x
The benchmark started with 1,756 cases spanning all 207 operations. The comparison kept 809 cases where the two implementations returned matching outputs and produced reliable timings. On those cases, Hugging Face recorded 629 wins, 176 losses, and four ties against ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a on an Apple M4 GPU.
The geometric mean speedup was 2.57x and the median was 1.90x. Four examples ranged from 1.14x for MatMul to 3.52x for Add. An unusual bilinear Einsum case exceeded 10,000x because the general implementation hit a severe slow path. The authors explicitly say such outliers are not the speedup an application should expect everywhere.
The comparison stops at individual operations rather than complete models. Its clock covers work on the GPU, excluding kernel loading, session creation, input upload, shader compilation, and output readback. Short operations are harder to time and can benefit from GPU cache state.
The operation benchmark has a clearly stated boundary. It is not a deployment benchmark. The retained 809 cases compare performance where both sides agreed and could be timed reliably. Before applying the aggregate to a particular model, investigate why the other 947 starting cases sat outside the comparison.
This is the same discipline we applied to FuXi's public runner: report the evaluated slice, the excluded slice, and the claim each slice can support.
Put a clock around the user's wait
Begin with the actual model and interaction. Pick inputs that resemble production traffic. Measure a cold path and a warm path rather than relying on an average across unrelated operations.
Start the cold clock before the package or kernel is cached and stop it at the first usable result. This captures artifact download, device acquisition, shader compilation, model loading, input transfer, dispatch, and readback. Repeat the workload after initialization for a warm measurement. Do not average the two. In an occasionally used product, cold work may dominate what people experience.
Record p50 and p95 latency across the browsers, operating systems, GPUs, and drivers that appear in real traffic. The Transformers.js WebGPU guide warns that availability depends on all four and that experimental behavior is especially visible outside Chromium. A feature check such as "gpu" in navigator is only the beginning. The application must also record device acquisition failures, compilation errors, incorrect outputs, timeouts, and the path selected when WebGPU is unavailable.
Latency needs a correctness result beside it. Compare representative outputs with a trusted implementation and choose tolerances for each data type and operation. A faster kernel can alter a ranking, transcript, embedding neighborhood, or generated token distribution without producing a visible crash.
For server-backed products, compare the browser path with the full remote request, including network latency and queueing. A shared endpoint such as api.ish.chat can remain a fallback while local execution is tested. Do not blend local and remote samples into one latency chart. Record which path served each response, a lesson reinforced by RuBench's fallback-model finding.
Record the artifact that actually ran
Because the loader downloads kernel packages from Hub repositories, the repository ID and contract version belong in each result. Save the metadata digest, npm package version, browser version, and GPU adapter too. A regression report without this tuple may compare different artifacts while appearing to repeat one test.
The repository format makes this practical because the contract, provenance, tests, benchmark inputs, and WGSL templates travel together. Review those files as executable dependencies. Cache only artifacts whose digest matches the expected metadata, and keep a known-good revision available for rollback.
Fleet runs correctness and performance checks in the browser, giving Hugging Face coverage across more devices. With consent, each run contributes private evidence for finding device-specific failures and tuning variant selection. That helps the maintainers. It does not test a product's model, audience, privacy requirements, or fallback policy.
A practical rollout
Pin @huggingface/kernels to an exact preview version and record the Hub metadata used in the build. After running the shipped correctness cases, add cases for the shapes and data types produced by your model.
Test cold and warm end-to-end latency on a modest device before moving to a flagship laptop. Include unsupported browsers and force WebGPU failures so the fallback path runs during the test. Compare output quality as well as speed. Log which implementation served every request.
Watch resource use through a long session as well. The WebGPU specification says browsers may limit GPU memory, use watchdog timers to preserve responsiveness, throttle background work, and warn about heavy power use. A short operation win can coexist with a tab that becomes unpleasant after repeated inference.
The collection gives browser AI developers inspectable contracts, tests, and shader implementations. Its benchmark shows that many operations can be faster. Any claim about a finished product still needs timings for the download, compilation, transfers, complete model, and fallback.
