A browser VAD without an inference runtime
The standard way to run a voice activity detector in a web page is to ship a general-purpose inference runtime and a model file, then ask the runtime to execute the model. It works. It also costs 5.5 MB, 200 ms of start-up and a main-thread callback per frame. We wanted to know what the same model costs when you write its six operations down by hand.
The model is small; the runtime is not
Silero VAD v6 is a windowed FFT, four small convolutions, one LSTM cell and a sigmoid. The weights are 974 KB as float32. The runtime that the popular browser package uses to execute them is 3.6 MB of WebAssembly on its own, and the ONNX file carries a 264 KB matrix that is nothing more than a Hann-windowed DFT basis, because the exporter did not know it could compute one.
So we wrote the forward pass in Rust. Reflect pad, 256-point radix-2 FFT at hop 128, the four conv blocks with their strides, the LSTM gates in PyTorch order, the 1x1 conv, the sigmoid. Then we compared the output to onnxruntime-web on the same clip: maximum difference 7.8e-7. Same model, same answer.
What it bought
| Auricle | onnxruntime-web, as vad-web runs it | |
|---|---|---|
| Per 32 ms frame, p50, Chromium | 0.052 ms | 0.182 ms |
| Per frame, WebKit | 0.050 ms | 0.184 ms |
| Start-up to a ready model | 13 to 28 ms | 175 to 215 ms |
| Download, gzipped | 925 KB | 5,520 KB |
| Files to copy into your app | none | model, runtime, worklet |
Three things did most of the work. Replacing the basis matmul with an FFT removed 264k of the roughly 720k multiply-adds per frame. Laying activations out time-major so every inner loop is one contiguous dot product, then writing that dot product with explicit SIMD lanes, took the frame from 0.19 ms to 0.05 ms. And putting the scratch buffers in the struct means the audio path never allocates.
The speed does not change detection latency, which is set by the frame size and the confirmation policy. It changes what is possible around the detector: the decision runs in a worker, the duck command goes straight to the playback worklet over a MessagePort, and a phone has headroom left for the rest of the page.
What it did not buy
Accuracy. It is the same model, so it is exactly as accurate as before: 93.7% frame F1 on TEN VAD's human-labelled recordings, where TEN VAD itself scores 93.1% at the same threshold. We tried the usual post-processing tricks, smoothing and hysteresis and a 16 ms hop, and every one of them lowered the score. That is in the benchmarks page too, because negative results are the ones people repeat.
Reproduce it: ./scripts/build-wasm.sh && cd bench && npm install && npm run bench. The numbers above are from the benchmarks page, with the browser matrix and the TEN set alongside.