Blog

Whisper in a browser tab now takes about half the download

Bar chart of download size for four Whisper model sizes. large-v3-turbo: 759 MB before, 493 MB packed for all languages, 412 MB packed for English. small: 661, 183 and 171 MB. base: 187, 55 and 47 MB. tiny: 92, 28 and 25 MB.
What the browser downloads for each Whisper model size: the files the page used before, and the two packed builds.

Anything to Text is Oskar's page for turning files into text without uploading them. For audio and video it runs Whisper, OpenAI's speech-to-text model, inside the tab. As of today the largest Whisper model on that page is a 412 MB download where it was 759 MB, and the smaller models are about a quarter of their old size. On English the transcripts are as accurate as the full-precision model's, and they arrive about one and a half times as fast as before.

The model is stored as small integers and run by our own WebGPU code

The page used to run Whisper through transformers.js, a library that loads models in the ONNX format. Those files hold the smaller models as 16-bit and 32-bit numbers, and the largest at 4 bits in a layout that still comes to 759 MB. The new files, which we call packs, store each weight as an integer of 4 to 7 bits with a shared scale factor, and they are served gzipped. Five bits a weight comes out smaller than the old 4-bit files because integers that cluster near zero compress well and the pack stores the vocabulary table once.

No existing library reads that format, so the page now carries its own engine: we wrote the matrix multiplies, the attention, and the decoder that emits one word-piece at a time as programs for the GPU. It is the approach that put Datalab's OCR model in a browser tab on November 18, applied to a model about 70 times larger. Given the same packed weights, the engine's output matched the reference implementation from Hugging Face on every test clip, word-piece for word-piece; we checked timestamps on the smallest model. We did not try re-exporting the ONNX files to the same sizes.

English gets the smallest build and other languages get their own

The weights squeeze further for English than for other languages. With the largest model at 5 bits per weight, the share of English words transcribed wrongly moved by 0.04 percentage points, which is inside the noise of the test. On Spanish, German, French and Norwegian the same pack got 0.8 points worse.

Oskar's answer was to ship both: "We should have two versions for the page, a default optimized one for English and regular as option". The English-optimized build is the default. The all-languages build keeps more bits where the other languages need them, costs 493 MB for the largest model, and tested level with the unpacked model on those four languages. The page switches to it by itself when a language other than English is selected or when translation is asked for. The three smaller models also use it when the language is left on auto-detect, because their English builds are OpenAI's English-only models.

It transcribes about one and a half times as fast and takes 815 MB on the GPU

On Oskar's Mac, with the page's own settings on both setups, a seven-second recording took 0.8 seconds to transcribe with either packed build and 1.3 seconds with the old one. A 28-second recording took 1.6 to 1.7 seconds against 2.3 to 2.7. The two measurements were made on one machine.

Only the download is smaller. Once loaded, the engine keeps each weight in a full byte on the GPU, so the largest model occupies 815 MB there. We have not yet kept the bits packed on the GPU, which would shrink that.

The packed models need WebGPU, and we have tested them in Chrome only. In a browser without it the page runs Whisper the old way, and it falls back to the old files if a pack fails to load.

Smaller models did much of the surrounding work

Oskar asked that the work be delegated to lesser models where it could be. A Claude Code session on the web ran it. It handed pieces to Sonnet and Haiku, Anthropic's smaller models: they surveyed what already existed, wrote the harness that measures accuracy, the code that turns audio into a spectrogram, the word-piece decoder and the demo page, and read the writeup cold to say what did not make sense. The session wrote the engine and chose the builds. The accuracy runs for the larger models went to free Kaggle GPUs, and the speed figures came from a tab left open on Oskar's Mac.

There is a standalone demo with a record button, and the writeup has the accuracy tables for every packing scheme tried. Several of them failed.